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REMARKS 

In view of the above amendments and following remarks, reconsideration is respectfully 
requested. 

Concurrently submitted herewith is a letter Re: Proposed Drawing Amendment along with 
Figures 1 6 marked in red to make minor changes and with Figures 1 8-20 marked in red to be labeled 
as "Prior Art" as required by the Examiner in paragraph 5 on page 3 of the Office Action. The 
Examiner's approval for such changes is respectfully requested. 

By this amendment, the specification and abstract have been reviewed and revised so as to 
make editorial changes. Enclosed herewith is a marked up version of the specification and abstract 
entitled "Version with Markings to Show Changes Made". 

Further by this Amendment, claims 4, 5, and 8-21 have been canceled, claims 1-3 and 6-7 
have been amended, and claims 22-34 have been newly added. Accordingly, it is submitted that 
claims 1-3, 6-7, and 22-34 are currently pending in this application. It is submitted that no new 
matter has been added. It is noted that newly added claims 23-28 are directed towards a recording 
medium containing program code for causing a computer to perform the method of claims 1 -3, 6-7, 
and 22, respectively, and that claims 29-34 correspond to an apparatus operable to perform the 
method of claims 1-3, 6-7, and 22, respectively. Accordingly, it is submitted that each of newly 
added claims 22-34 read on the previously elected species. 

It is noted that the Examiner has rejected claims 1-3 and 6-7 under 35 U.S.C. § 102(a) as 
being anticipated by the admitted prior art for the reasons contained in paragraph 8 on pages 3-4 of 
the Office Action. 

While the Applicants traverse the aforementioned art rejection, by this Amendment, each of 
independent claims 1 and 7 have been amended so as to more clearly distinguish the present 
invention from the prior art in an effort to expedite allowance of this application without intending 
to acquiesce to the Examiner's aforementioned prior art rejection. 

Accordingly, it respectfully is submitted that the present invention, at least as embodied by 
newly amended independent claims 1 and 7 and newly added independent claims 23, 28, 29, and 34, 
clearly patentably distinguishes over such the prior art for at least the following reasons. 
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First, the Applicants would like to point out that the Examiner relies on Figures 1 8-20 (Prior 
Art) of the present application as allegedly disclosing the claimed features of the present invention. 
Particularly, the Examiner essentially asserts that a padding process for the small images spaces as 
claimed and discussed in the present application is disclosed by the admitted prior art shown in 
Figures 18-20. 

However, the Applicants would like to strongly emphasize that the present invention, as 
clearly reflected by Figures 1, 2, 3(a)-3(c), provides a method of dividing an original image space 
(30 1 as shown in Figure 2, for example) into plural small images spaces (40 1 -404 as shown in Figure 
2, for example) before the padding process. The admitted prior art, on the other hand, provides a 
method for dividing the original image space to plural small images spaces (as shown in Figure 20) 
after t he padding process of the original image space (as shown in Figures 18-19). 

Particularly, as clearly recited in each of independent claims 1, 23, and 29, the present 
invention provides a pixel rearrangement in which plural pixels in the original image space are 
grouped to form plural small image spaces each comprising pixels of the same group so that a pixel 
value correlation within the small image spaces becomes higher than a pixel value correlation within 
the original image space, and then performs pixel padding in which values of insignificant pixels in 
each of the small image spaces obtained by the pixel rearrangement are replaced with padding pixel 
values generated on the basis of the significant pixel values in each of the small image spaces. 

Thus, the present invention is able to prevent high frequency components of image signals 
forming the original image space from increasing due to a padding process of the image space by 
performing the padding process to the smaller image spaces which have higher pixel value 
correlation than that of the original image space. 

It is strongly submitted that the above discussed features of the present invention are 
encompassed within the limitations of independent claims 1, 23, and 29 of the present application. 
Further, it is submitted that the above limitations, and the above advantages resultant therefrom, are 
not disclosed or suggested by the admitted prior art cited by the Examiner. 

Particularly, as mentioned above and as clearly evident from Figure 1 8 and 20, the admitted 
prior art method performs the padding process directly to the original and undivided image space. 
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It is only after such padding process of the original image space, when the admitted prior art method 
performs the division of the original image space to smaller image spaces as clearly shown in Figure 
20. Thus, the admitted prior art clearly does not perform a padding process to small images spaces 
which are generated by dividing an original image space, as clearly recited in each of independent 
claims 1, 23, and 29 of the present application. 

Next, the present invention, as clearly recited in each of independent claims 7, 28, and 34 of 
the present application, provides a method of dividing the original image space into a first small 
image space comprising pixels on odd-numbered pixel rows in the original image space and a second 
small image space comprising pixels on even-numbered pixel rows in the original image space, and 
then generating first padding pixel values from values of significant pixels in the first small image 
space and replacing values of insignificant pixels in the first small image space with the first padding 
pixel values, and generating second padding pixel values from values of significant pixels in the 
second small image space and replacing values of insignificant pixels in the second small image 
space with the second padding pixels values. 

By providing the aforementioned features, it is possible to prevent high frequency 
components from being brought into the image signal even for a padding process for an interlaced 
image which has a large motion. 

It is strongly submitted that the above discussed features of the present invention are 
encompassed within the limitations of independent claims 7, 28, and 34 of the present application. 
Further, it is submitted that the above limitations, and the above advantages resultant therefrom, are 
not disclosed or suggested by the admitted prior art cited by the Examiner. 

Particularly, the method of the admitted prior art introduces high frequency components into 
an image signal through the padding process when the image signal is an interlaced signal having 
large motion due to the fact that the compensation process is performed directly to a frame, as 
mentioned earlier and as shown in Figures 1 8- 1 9, so as to include pixels of a first field and a second 
field corresponding to an interlaced signal. Thus, the admitted prior art clearly does not divide the 
original image space into a first and second small image spaces comprising pixels on odd-numbered 
and even-numbered pixel rows, respectively, and then subject the small image spaces to a padding 
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process for replacing pixel values of the insignificant pixels with padding pixel values obtained 
based on pixels values of the significant pixels, as clearly recited in each of independent claims 7, 
28, and 34 of the present application. 

For the foregoing reasons, it is submitted that independent claims 1, 7, 23, 28, 29, and 34, 
as well as claims 2-3, 6, 22, 24-27, and 30-33 dependent therefrom, clearly are allowable. 

In view of the foregoing, it is submitted that the present application now in fact clearly is 
in condition for allowance and the Examiner therefore is requested to pass this case to issue. 

In the event, however that the Examiner has any comments or suggestions of a nature 
necessary to place this case in condition for allowance, then the Examiner is requested to contact 
Applicants 1 undersigned attorney by telephone to promptly resolve any such matters. 

Respectfully submitted, 
Choong Seng BOON 

By : ^Cl^&SSZ— 
Dhiren R. Odedra 
Registration No. 41,227 
Attorney for Applicant 

DRO/lgs 

Washington, D.C. 20006-1021 
Telephone (202) 721-82^0 
Facsimile (202) 721-8250 
October 17, 2001 
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Version with Markings to 
Show Changes Made 



SPECIFICATION 
GE PADDING METHOD, IMAGE PROCESSING APPARATUS , 
AND DATA RECORDING MEDIUM 

t 

Technical Field 

The present invention relates to a digital image padding 
method, an image processing apparatus, and a data recording 
medium and, more particularly to a padding method in which 
padding of pixels values is performed to a digital image signal 
corresponding to an interlacing scanned image having an 
arbitrary shape, an image processing apparatus for coding or 
decoding the digital image signal using this padding method, 
and a data recording medium which contains a program for 
implementing image signal processing by the padding method 
or the image processing apparatus in a computer. 

Background Art 

In order to store or transmit digital image information 
efficiently, it is required that digital image information 
be compressive^ coded. In the status quo, as a method for 
compressively coding the digital image information, there 
are waveform coding methods such as subband, wavelet, fractal, 
and so forth, as well as DCT (Discrete Cosine Transform) 
typical of JPEG (Joint Photographic Coding Experts Group) 
or MPEG (Moving Picture Experts Group) . 
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Meanwhile, as a method for eliminating redundant image 
information between adjacent frames or the like, there is 
a method in which inter-frame prediction is performed using 
motion compensation by representing values of pixels of a 
current frame by difference values between these values'and 
values of pixels of a previous frame, and a difference image 
signal of the difference values is subjected to waveform 

coding. ► 

An interlacing scanned image signal such as a video 
signal in a current television forms an interlaced image. 
In this interlaced image, one frame is composed of o<|d- 
numbered and even-numbered fields, the scanning timings of 
which differ from each other. For this reason, in the 
interlaced image, whether or not correlation of pixel values 
between adjacent scanning lines is high depends on largeness 
of motion of each object displayed on a frame (each object 
having an arbitrary shape in a frame image) . 

Accordingly, when coding an image signal corresponding 
to the interlaced image, in a case where there is high 
correlation of pixel values between scanning lines, motion 
compensation and waveform coding are performed to the image 
signal frame by frame, while in a case where there is low 
correlation of pixel values between scanning lines, the image 
signal is divided into odd-numbered and even-numbered fields, 
the image signals of which are subjected to motion 



compensation and waveform coding field by field. 

In recent years, for reproducing an image signal for 
each of objects in a frame image while increasing efficiency 
in compressing an image signal, a coding method has been 
conceived, wherein image signals of the objects are * 
compressively coded and transmitted separately for ^ach 
object- A coded image signal which has been coded by this 
coding method is decoded at the reproducing end in a manner 
adapted to the coding method. Specifically, in thi^ 
decoding, coded image signals of respective objects are 
decoded, and the resulting decoded image signals of ^ 

I 

respective objects are synthesized, to produce a reproduced 
image signal. Then, based on the reproduced image signal, 
a frame image comprising respective objects is displayed. 

By using the coding method for performing coding to the 
image signal for each object as described above, it becomes 
possible to combine objects freely to generate a composite 
image, whereby moving images can be reedited with ease. In 
addition, a moving, image comprising only objects with high 
importance can be displayed without reproducing objects with 
relatively low importance, depending on^degree of congestion 
of a communication path, performance of a reproducing 
apparatus, or preference of a viewer. 

For coding an image signal of an object, (an image having 
an arbitrary shape ), waveform transform which performs 
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signal processing adapting to a shape of the object, for 

example, shape adaptive DCT, or waveform transform to a 

padded image signal, is employed. In the method, 

specifically, padding is performed to an image signal forming 

an image space (rectangular region) by replacing values of 

pixels in its insignificant region with padding pixel values 

obtained by a prescribed method, and then^convent iona 1 8 x 

8 cosine transform is performed to the padded image signal. 

) 

The insignificant region is a region outside an object in 
the rectangular region, and comprises pixels which have no 
values for displaying the object. In other words, an jmage 
signal of the insignificant region comprises so-called 
insignificant sample values. Also, the 8x8 cosine 
transform is v waveform transform which performs cosine 
transform to the image signal of the rectangular region for 
each image space comprising 8x8 pixels. 

As'a method for eliminating redundant signal between 
adjacent frames, there is a method for obtaining difference 
between an image signal of a target macroblock to be coded 
and its prediction signal for each image space (macroblock) 
comprising 16 x 16 pixels, as a prediction error signal 
(difference signal) . Here, it is assumed that the 
prediction signal is an image signal of jtfi prediction region 
obtained by motion compensation. The motion compensation 
process detects a region comprising 16 x 16 pixels 
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corresponding to an image signal in a coded or decoded frame 

where difference between the image signal of the target 

macroblock and the image signal is the smallest, as a 

prediction region. 

In some cases, however, the prediction region £lso 

includes pixels (hereinafter referred to as insignificant 

pixels) which have insignificant sample values. In such 

cases, when the difference between the prediction signal of 

I 

the prediction region including insignificant values and an 

image signal of the target region is computed, difference 

J 

values tend to be very large, since the sample values cjf the 
insignificant pixels are not always optimal prediction 
values for smaller difference. 

As a solution to this, padding is performed to the image 
signal of the prediction region by replacing insignificant 
sample values by prescribed padding values, and then 
difference between the padded prediction signal and the image 
signal of the target macroblock is computed as a difference 
signal (prediction error signal), which is subjected to 
transformation for coding. This Padding of the prediction 
signal can suppress the difference signal. 

In the prior art pixel value padding, as a padding value 
for an insignificant sample value , sample values of 
significant values (pixels having significant sample values) 
adjacent to the corresponding insignificant pixel in the 



vertical and horizontal directions, respectively, are 
averaged, and the resulting average is used. By the use of 
the average of the sample values of the significant pixels, 
high-frequency components into an image signal (prediction 
signal) in a padded image space can be suppressed, thereby 
increasing coding efficiency- I 

However, if padding which uses the average of the 
significant pixels adjacent to the insignificant pixels in 

f 

the vertical and horizontal directions as the padding value 

is applied to the image signal of the interlaced image, 

I 

high-frequency components of the image signal increase,, 
which will be described below in detail. 

In the interlaced image, especially in a case where 
motion of an object is large, there is low correlation 
of pixel values between adjacent scanning lines. Figure 18 
schematically shows an example of arrangement of pixel values 
which shows J^Ja-cfs^low correlation of pixel values between 
adjacent scanning lines in the interlaced image, as 
arrangement of pixel values in an image space 301. 

The image space 301 is an image space comprising 8 x 
8 pixels. In the figure, each pixel is represented as a 
square. Pixels (for example pixels 303 and 304) of hatched 
squares are pixels (significant pixels) having significant 
sample values, and pixels (for example pixel 302) of squares 
which are not hatched are pixels (insignificant pixels) 
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having insignificant pixels. The number of each square 
indicates a sample value of the corresponding pixel. 

Since scanning timings differ from each other between 
odd-numbered and even-numbered fields in the interlaced 
image, in a case where motion of an object in an image is 
large, it looks like the object has a shape of two outlines 
in a frame composed of these two fields, as it can be observed 
in the image space 301 in figure 18. In this case, ^for 
example, the significant pixel 304 is adjacent to its upper 
and lower insignificant pixels. p&t/dfj 

Next, a prior art padding method. and its Raiding result 
will be described with reference to figures 18 and 19. 

In the prior art padding method, three steps are 
performed . 

In a first step SI, sample values of insignificant 
pixels are replaced with sample values of significant pixels 
adjacent thereto in the image space 301, for insignificant 
pixels aligned in the horizontal direction, in the order from 
inward to outward of the image space 301. Then, sample 
values of insignificant pixels in the image space to which 
this horizontal replacement process has been performed are 
replaced with sample values of significant pixels adjacent 
thereto, for insignificant pixels aligned in the vertical 
direction, in the order from inward to outward of the image 
space. Process in the first step SI thus performed to the 
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image space 301 results in an image space 350 comprising 
pixels having sample values shown in figure 18. 

More specifically, in the horizontal replacement 
process in the first step SI, sample values of insignificant 
pixels on pixel rows 354-359 in the* image space 350 'are 
replaced with padding values (sample values of significant 
pixels) . In addition, in the vertical replacement process 
in the first step SI, sample values of insignificant pixels 
on pixel rows 352. and 353 are replaced with sample values 
of significant pixels and padded pixels (insignificant 
pixels whose sample values have been replaced with padding 
values) on the pixel row 354. 

In a second step S2, in the reversed order of the first 
step SI, sample values of insignificant pixels are replaced 
with sample values of significant pixels adjacent thereto 
in the image space 301, for insignificant pixels aligned in 
the vertical direction, in the order from inward to outward 
of the image space. Then, sample values of insignificant 
pixels are replaced with sample values of significant pixels 
adjacent thereto in the image space to which vertical 
replacement process has been performed, for insignificant 
pixels aligned in the horizontal direction, in the order from 
inward to outward of the image space. Process in the second 
step S2 thus performed to the image space 301 results in an 
image space 351 comprising pixels having sample values shown 



in figure 18 . 

More specifically, in the vertical replacement process 
in the second step S2, sample values of insignificant pixels 
on pixel columns 362-365 are replaced with padding values 
in the image space 351. In addition, in the horizontal 
replacement process in the second step S2, sample values of 
insignificant pixels on pixel columns 360 and 361 are 
replaced with sample values of significant pixels and padded 
pixels on the pixel column 362, and likewise, sample values 
of insignificant pixels on pixel columns 367 and 368^ are 
replaced with sample values of significant pixels and padded 
pixels on the pixel column 365. 

In a third step S3, as shown in figure 19, sample values 
of pixels in the image space 350 obtained through the first 
step SI and the corresponding pixels in the image space 351 
obtained through the second step S2 are averaged, which 
results in an image space 380 comprising pixels having sample 
values in figure 19. 

Resampling which divides pixels forming the image space 
380 into pixels corresponding to odd-numbered and 
even-numbered fields^on assumption that the image space 380 

las* 

is a frame, results in an image space 381 comprising plural 
pixels corresponding to the odd-numbered field and an image 
space 382 comprising plural pixels corresponding to the 
even-numbered field as shown in figure 20. In the image 
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spaces 381 and 382 corresponding to these respective fields, 
sample values of pixels are non-uniform, which introduces 
high-frequency components into image signals in image spaces 
381 and 382. 

In addition, this problem associated with the padding 

occurs in the non-interlacing scanned image (progressive 

image) as well as in the interlacing scanned image 

(interlaced image). Specifically, in many cases, the 

"f 

progressive image has a stripe pattern, and for the case of 
the progressive image having the stripe pattern, the above 
resampling is performed in such a manner that plural pixels 
forming a frame (image space) are collected into a stripe 
portion and the other portion in the image space, to form 
image spaces corresponding to the stripe pattern portion and 
the other portion, and then the resulting resampled image 

spaces are coded. 

In'this case, if padding is performed to the image signal 
of the progressive image having the stripe pattern by the 
prior art padding method, high-frequency components are 
introduced into an image signal corresponding to the 
resampled image space, leading to degraded coding 
efficiency . 

Further, >Ke similar problem occurs in padding methods 
other than the above, which computes padding values using 
significant sample values adjacent to insignificant pixels 




in the horizontal or vertical direction. 

The present invention is directed to solving the 
above-mentioned problem and has an object to provide a 
digital image padding method in which padding is performed 
to an interlacing scanned image or an image of a stripe pattern 
having an arbitrary shape while suppressing degradation of 
efficiency in coding the prediction error signal, an image 
processing apparatus which performs image coding or image 
decoding by the padding method, and a data recording medium 
which contains a program for implementing these image 
processing by the padding method or the image processing 
apparatus in a computer. 

DisxJre^ure of the Invention 

According to gJUim-*of the present invention, a digital 
image padding method er£H^-e-p^-^^ for performing 

padding 'of pixel values to a digital image signal forming 
an original image space comprising an image having an 
arbitrary shape and comprising significant pixels and 
insignificant pixels, comprises a pixel rearrangement step 
in which plural pixels in the original image space are grouped 
according to a prescribed rule, to form plural small image 
spaces each comprising pixels of the same group; and a pixel 
padding step in which values of insignificant pixels in each 
small image space are replaced with padding pixel values 



generated by a prescribed method. 

In the digital image padding method thus constructed, 
plural pixels forming the original image space are extracted 
according to a prescribed rule, to form plural small images. 
In other words, it is possible to divide the original iirfage 
space where there is low correlation of pixel values between 
adjacent pixels or scanning lines into plural small images 
where there is high correlation of pixel values between the 
adjacent pixels or scanning lines. Accordingly, padding is 
performed to the interlacing scanned image or the image of 



a stripe pattern having an arbitrary shape for each snjal 1 
image space with high correlation of the pixel values rather 
than the original image space with low correlation of pixel 
values between adjacent pixels or scanning lines. In this, 
case, in the padded small image, there is high correlation 
of the pixel values, and it is possible to avoid introduction 
of high-frequency components into the image signal 
corresponding to the original image space in process for 
padding the interlacing scanned image or the image of a stripe 
pattern having an arbitrary shape. 

As a result, differential coding or differential 
decoding is performed to the interlacing scanned image while 
suppressing degradation of coding efficiency resulting from 



According to c rlaii rr 2 of the present invention, in the 



the padding. 
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digital image padding method of G^ra-tftt~~l , in the pixel 

rearrangement step, sampling is performed by extracting 

pixels every (N + l)-th (N: positive integer) pixel in a 

prescribed direction of the image space (N + 1) times by using 

the first to the (N + l)-th pixels as starting sample pixels, 

to form (N + 1) pieces of small image spaces each comprising 

a prescribed number of pixels obtained by one sampling. 

In the digital image padding method thus constructed, 

) 

pixels in the original image space are extracted at intervals 
of a prescribed number of pixels in a prescribed direction 
of the original image space, to form the small image spacej. 
Therefore, it is possible to divide the original image space 

comprising an image having an arbitrary shape of a vertical 

) 

stripe pattern or a horizontal stripe pattern into small 
image spaces where there is high correlation of pixel values 
between adjacent pixel rows or between adjacent pixel columns, 
irrespective of a width or arrangement spacing of stripes. 

According t-o c 1. a i of the present invention, in the 

digital image padding method of eia4rffl~2-, operation values />^.^ 
obtained by performing operation on values of significant 
pixels in an N-th (n=l, 2,..., N + l) small image space are used 
as the padding pixel values for replacing the values of 
insignificant pixels in the n-th small image space. 

In the digital image padding method thus constructed, 
values of insignificant pixels in a small image space are 
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replaced with padding pixel values obtained from values of 
significant pixels in the same small image. Therefore, it 
is possible to suppress generation of pixel values which are 
significantly different from values of significant pixels 
in the small image space in padding process. * 
According to claim - 4 of the present invention, thej 
digital image padding method of - claim 2 further comprises r 
an image space restoring step in which plural pixels forming 
padded small image spaces are rearranged following a rule 
adapted to the prescribed rule of the grouping so that these 

j 

pixels forms a restored image space which has the same pix^l 
rearrangement as the original image space. 

In the digital image padding method thus constructed, 
after the original image space is divided into plural small 
images where there is high correlation of pixel values, and 
padding of pixel values is performed to these small image 
spaces, pixels in the padded small image space are rearranged 
so that they have the same arrangement as the original image 
space. Therefore, the image signal corresponding to the 
padded original image space can be processed in the same 
manner that the image signal corresponding to the unpadded 
original image space is processed. 

According to claim 5 of the present invention, in the 
digital image padding method o f -e-3r«nr-4 , operation values "IC^*-^^ 
obtained by performing operation on values of significant 




pixels in an n-th (n-1, 2,..., N + l) small image space are used 
as the padding pixel values for replacing the values of 
insignificant pixels in the n-th small image space- 
In the digital image padding method thus constructed, 
values of insignificant pixels in a small image space are 
replaced with padding pixel values obtained from values <^f 
significant pixels in the same small image space. Therefore, 
it is possible to suppress generation of pixel values which 

^ 

are significantly different from values of significant ; 
pixels in the small image space in padding process. 

According to t.laim 6 of the present invention, in the 
digital image padding method of claim 1, in the pixel / tfis&*<^''f J +? 
rearrangement step, sampling is continuously performed twice 
using the first and second pixel rows as starting sample pixel 
rows, by extracting pixels on every other pixel row in the 
vertical direction of the original image space, to form first 
and second small image spaces each comprising a prescribed 
number of pixels obtained by the first and second samplings, 
respectively. 

In the digital image padding method thus constructed, 
every other pixels are extracted in the vertical direction 
of the original image space, to form a small image space 
composed of odd-numbered scanning lines in the original image 
space and a small image space composed of even-numbered 
scanning lines and a small image. space composed of even- 
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numbered scanning lines. Therefore, padding of the 
interlacing scanned image is performed for each small image 
space corresponding to each field where there is high 
correlation of pixel values between adjacent scanning lines 
rather than the original image space where there is low* 
correlation of pixel values between them. As a result, ^t 
is possible to avoid introduction of high-frequency 
components into an image signal of the interlacing scanned 

\ 

image in padding process. 

According to c-ta-irflb^T-of the present invention, a digital 

j 

image padding method for performing padding of pixel values 
to a digital image signal forming an original image space 
comprising an image having an arbitrary shape and comprising 
significant pixels and insignificant pixels, comprises the 
steps of dividing the original image space into a first small 
image space comprising pixels on odd-numbered pixel rows in 
the original image space and a second small image space 
comprising pixels on even-numbered pixel rows in the original 
image space; and generating first padding pixel values from 
values of significant pixels in the first small image space 
and replacing values of insignificant pixels in the first 
small image space with the first padding pixel values, and 
generating second padding pixel values from values of 
significant pixels in the second small image space and 
replacing values of insignificant pixels in the second small 



image space with the second padding pixel values. 

In the digital image padding method thus constructed, 
padding of the interlacing scanned image is performed for 
each of first and second small image spaces corresponding 
to fields where there is high correlation of pixel values 
between adjacent scanning lines rather than the origina|l 
image space where there is low correlation of pixel values 
between them. As a result of this, it is possible to avoid 
introduction of components into an image signal of the 
interlacing scanned image in padding process. 

! 

Besides, values of insignificant pixels in each smaj.1 
image space are replaced with padding pixel values obtained 
from values of significant pixels in the same small image 
space, and it is therefore possible to suppress generation 
of pixels values which are significantly different from 
values of significant pixels in the small image space in 
padding 'process . , 

According to G - laim 4of the present invention, a digital 
image padding method for performing padding to a digital 
image signal forming an original image space comprising an 
image having an arbitrary shape and comprising significant 
pixels and insignificant pixels, by replacing values of 
insignificant pixels with padding pixel values determined 
from values of significant pixels in the original image space, 
wherein, significant pixels which have values used for 
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determining the padding pixel values are pixels other than 

pixels adjacent to insignificant pixels to be padded. 

In the digital image padding method thus constructed, 

the padding pixel values for replacing values of 

insignificant pixels are determined from values of 

significant pixels other than pixels adjacent to the ] 

insignificant pixels in the original image space, and 

therefore the values of the insignificant pixels are not 

I 

replaced with the values of the pixels adjacent thereto. As 
a result, it is possible to perform padding of pixel values 
to the original image space where there is low correlation 
of pixel values between adjacent pixels or scanning lines 
while suppressing introduction of high-frequency components 
into the image signal corresponding to the original image 

SPaCS " ^ /fs^S^ 

According to ©^a4rm--9^of the present invention, in the 
digital 'image padding method of - claim -8 , significant pixels^^^ 
which have values used for determining the padding pixel 
values are one pixel apart from the insignificant pixels to 
be padded. 

In the digital image padding method thus constructed, 
values of insignificant pixels on an odd-numbered scanning 
line are replaced with padding pixel values obtained from 
values of significant pixels on another odd-numbered 
scanning line in the original image space, and values of 
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insignificant pixels on an even-numbered scanning line are 
replaced with padding pixel values obtained from values of 
significant pixels on another even-numbered scanning line 
in the original image space. Therefore, it is possible to 
avoid introduction of high-frequency components into th& 
image signal corresponding to the interlacing scanned imag^e 
in padding process. y 

According to -el ai m lO of the present invention, an image 
processing apparatus for performing image coding which 
comprises correlation identif ying^jne-arns for identifying 

i 

correlation between pixel values of a digital image signal 
and outputting a sample identifier according to the 
identification result, a coding section for performing 
differential coding to the digital image signal by the use 
of a prediction image signal of the digital image signal 
according to the sample identifier and outputting a coded 
image signal, and a predicting section for producing the 
prediction image signal based on a decoded image signal in 
which the coded image signal has been locally decoded, the 
coding section comprises a subtracter for computing 
difference between the digital image signal and the 
prediction image signal as a difference signal; data 
compressing m^arfs for compressing the difference image 
signal to produce a compressed image signal; and a variable 
length encoder for performing variable length coding to the 
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compressed difference signal, and the predicting section 
comprises data expanding jrie-ans for expanding the compressed 
difference signal to produce an expanded difference signal; 
an adder for adding the expanded difference signal to the 
prediction image signal to produce a decoded image signal; 
padding TQ&ecfiS for performing padding according to the sample 
identifier by rearranging pixels in an image space formed 
by the decoded image signal and replacing values of 



insignificant pixels in the image space where the pixels have 
been rearranged with padding pixel values generated by a 
prescribed method; and prediction image signal producing 



padded decoded image signal according to the sample 
identifier . 

In the digital image processing apparatus thus 
constructed, the predicting section for producing the 
prediction image signal of the digital image signal, 
comprises the padding gie-ams for performing padding of pixel 
values to the decoded image signal in which the coded 
difference signal has been locally decoded in its 
differential coding process, and is used for producing the 
prediction image signal from the padded decoded image signal, 



the padding oie^ns first rearranging pixel values of the 
decoded image signal and then replacing the values of the 
insignificant pixels of the decoded image signal with the 




the prediction image signal from the 




prescribed padding pixel values. Therefore, when 

performing differential coding to the digital image signal 

of the interlacing scanned image or the image of a stripe 

pattern having an arbitrary shape, it is possible in padding 

process to avoid introduction of high-frequency components 

into the prediction image signal for use by differentia^ 

coding, differential coding can be performed to the 

interlacing scanned image while suppressing degradation of 

coding efficiency resulting from the' padding. 

<&/<&^^&? 

According to s- la - im L 3. of the present invention, in the 
image processing apparatus of ^. aim the coding section^^^ 

further comprises padding n^ans for performing padding to 
the difference image signal according to the sample 
identifier by rearranging pixels in the image space formed 
by the difference image signal and replacing values of 
insignificant pixels in the image space where the pixels have 
been rearranged with padding pixel values generated by a 
prescribed method, and the data compressing jp&je-a'hs performs 
information compression to the padded difference image 
signal to produce a compressed difference image signal. 

In the image processing apparatus thus constructed, 
pixel values of the difference image signal input to the data 
compressing ny^sfis are rearranged, and then padding is 
performed by replacing values of the insignificant pixels 
of the difference signal with the . prescribed padding pixel 
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values. Therefore, it is possible to avoid introduction of 
high-frequency components into the image signal input to the 
data compressing means. As a result, when performing 
differential coding to the digital image signal of the 
interlacing scanned image or the image of a stripe pattern 
having an arbitrary shape, waveform transform such as DC^T 
is performed with high efficiency. 

According to e laim 1 2 of the present invention, an image 

to 

processing apparatus for performing image decoding whicf 
comprises a reproducing section for reproducing a coded image 
signal in which a digital image signal has been coded by the 
use of a prediction image signal of the digital image signal 
and outputting a reproduced image signal, and a predicting 
section for producing the prediction image signal from the 
reproduced image signal, and the reproducing section 
comprise a data analyzing unit for analyzing the coded image 
signal to extract a compressed difference signal of the 
digital image signal and a sample identifier indicating 
correlation between pixel values of the digital image signal 
from the coded image signal; data expanding jrus-^ns for 
expanding the compressed difference signal to produce an 
expanded difference signal; and an adder for adding the 
expanded difference signal to the prediction signal to 
produce the reproduced image signal, and in the predicting 
section, using at least one of the reproduced image signal 
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and the prediction image signal obtained from the reproduced 

image signal as a signal to be padded, padding is performed 

to the signal to be padded according to the sample identifier, 

by rearranging pixels in an image space formed by the signal 

to be padded and replacing values of insignificant pixel's 

in the image space where the pixels have been rearranged wit^h 

padding pixel values generated by a prescribed method. 

^ In the image processing apparatus thus constructed, the 

) 

predicting section for producing the prediction image signal 
of the digital image signal comprises ^padding m^erfis for 
performing padding of pixel values to the reproduced image 
signal or the prediction image signal of the digital image 
signal as the signal to be padded, and the padding process 
rearranges pixel values of the signal to be padded and then 
replaces values of insignificant pixels of the signal to be 
padded with prescribed padding pixel values. Therefore, 
when decoding a coded difference signal of the interlacing 
scanned image or the image of a stripe pattern having an 
arbitrary shape, it is possible to avoid introduction of 
high-frequency components into the prediction image signal 
for use by the differential decoding. As a result, 
differential decoding adapted to differential coding which 
suppresses degradation of coding efficiency resulting from 
padding of pixel values, is realized. 
<^RSr~~ According to c- laim -3r3 of the present invention, in the 
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image processing apparatus of G-Jra- i-iu 12 , the predicting 
section further comprises ^padding m&afhs for performing 
padding to the reproduced image signal frame by frame or field 
by field according to the sample identifier, and a frame 
memory for storing a padded reproduced signal in which the 
reproduced image signal has been padded, and outputs thje 
padded reproduced image signal stored in the frame memory 
to the reproducing section as the prediction image signal. 

In the image processing apparatus thus constructed,, 
padding is performed to the reproduced image signal according 
to the sample identifier frame by frame or by field by field. 
Therefore, especially when performing differential decoding 
to the digital image signal of the interlacing scanned image, 
it is possible to avoid introduction of high-frequency 
component into the prediction image signal obtained from the 
reproduced image signal in padding process. 

According to -eirrirnr-3r4- of the present invention, in the 
image processing apparatus of c^ta-arm— T2 , the predicting y^^^r-v.,. 
section further comprises a frame memory for storing the 



V A ou- 
tproduced image signal, and ^padding jfte^ns for performing 

padding to the prediction image signal extracted from the 

reproduced image signals stored in the frame memory frame 

by frame or field by field according to the sample identifier, 

and outputs the padded prediction image signal to the 

reproducing section. 
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In the image processing apparatus thus constructed, 

padding is performed to the prediction image signal according 

to the sample identifier frame by frame or by field by field. 

Therefore, especially when performing differential decoding 

to the digital image signal of the interlacing scanned image*, 

it is possible to avoid introduction of high-frequency ^ 

components into the prediction image signal in the process 

for padding the prediction image signal. Besides, since 

) 

padding is performed only to the prediction signal fetched 
from the frame memory, amount of signals to be padded in the 

i 

predicting section can be reduced. ^ 

According to oie-irftHL-S- o f the present invention, an image 
processing apparatus for performing image decoding, which 
comprises a reproducing section for reproducing a coded image 
signal in which a digital image signal has been coded by the 
use of a prediction image signal of the digital image signal 
and outputting a reproduced image signal, and a predicting 
section for producing the prediction image signal from the 
reproduced image signal, the reproducing section comprises 
a data analyzing unit for analyzing the coded image signal 
to extract a compressed difference signal of the digital 
image signal and a sample identifier indicating correlation 
between pixel values of the digital image signal from the 
coded image signal; data expanding nj^ens for expanding the 
compressed difference signal to produce an expanded 
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difference signal; and an adder for adding the expanded 
difference signal to the prediction image signal to produce 
the reproduced image signal, and the predicting section 
comprises padding giS-ans for performing padding to the 
reproduced image signal according to the sample identifier*, 
by replacing values of insignificant pixels in an image spac^e 
formed by the reproduced image signal with padding pixels 
values generated from values of significant pixels other than 
pixels adjacent to the insignificant pixels; and ' 
rearrangement m^ns for rearranging pixels in an image space 
formed by the padded reproduced image signal, according to 
the sample identifier- f 

In the digital image processing apparatus thus 
constructed, when performing padding to the reproduced image 
signal obtained from the coded difference image signal 
containing digital image information, the padding pixel 
values for replacing values of insignificant pixels are 
determined from values of significant pixels other than 
pixels adjacent to the insignificant pixels in the image 
space formed by the reproduced image signal. Therefore, 
values of insignificant pixels will not be replaced with 
values of pixels adjacent thereto. As a result, when 
performing differential decoding to the digital image signal, 
it is possible to perform padding of pixel values in the 
original image space where there is low correlation of pixel 
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values between adjacent pixels or scanning lines while 

suppressing introduction of high-frequency components into 

the image signal in the original image space. 

& According to e - laim 15 of the present invention, a data 

recording medium for storing a program which makes a computer 

perform padding of pixel values to a digital image signa^l 

forming an original image space comprising an image having 

an arbitrary shape and comprising significant pixels and 

insignificant pixels, and the program makes the computer 

perform padding of the pixel values according to the digital 

image padding method of eirHrTTr-1 . ^^^^Jr^^^ 

I 

/ ^ By loading the program stored in the data recording 

medium thus constructed into the computer, padding is 
implemented by the computer without degrading coding 
efficiency when performing differential coding or 
differential decoding to the interlacing scanned image or 
the like. 

I g According to G±e^rrrr-3r^ of the present invention, a data 
recording medium for storing a program which makes a computer 
perform padding of pixel values to a digital image signal 
forming an original image space comprising an image having 
an arbitrary shape and comprising significant pixels and 
insignificant pixels, and the program makes the computer 
perform padding of pixel values according to the digital 
image padding method of e^fl . ^ ^ ^ ^ 



2S 



By loading the program stored in the data recording 
medium thus constructed into the computer, padding is 
implemented by the computer without degrading coding 
efficiency when performing differential coding or 
differential decoding to the interlacing scanned image. * 

According to - qlaim 1 - 9 - of the present invention, a datp 

recording medium for storing a program which makes a computer 

perform padding of pixel values to a digital image signal 

forming an original image space comprising an image having 

an arbitrary shape and comprising significant pixels and 

insignificant pixels, and the program makes the computer 

I 

perform padding of pixel values according to the digital 
image padding method of €laim -8 . ' ^<c* 
I By loading the program stored in the data recording 

medium thus constructed into the computer, padding is 
implemented by the computer without degrading coding 
efficiency when performing differential coding or 
differential decoding to the interlacing scanned image . 

According to claim 1 -9- of the present invention, a data 
recording medium for storing a program which makes a computer 
perform image signal coding, and the program makes the 
computer perform differential coding to a digital image 
signal by the image processing apparatus of . ^-y^ ^/.^ 

By loading the program stored in the data recording 
medium so constructed into the computer, differential coding 




of the interlacing scanned image or the like is implemented 
by software with no degradation of coding efficiency 
resulting from padding. . 

According to claim 2 0 of the present invention, a data 
recording medium for storing a program which makes a compute,r 
perform image signal decoding, and the program makes the 
computer perform differential decoding to the digital image 
signal by the image processing apparatus of <s-l a — -1-2 » £>f ffa^ 

By loading the program stored in the data recording 
medium thus constructed, differential decoding of the 
interlacing scanned image, which is adapted to differential 
coding with no degradation in coding efficiency resulting 
from padding, is implemented by software. 

According to o l a - im 21 of the present invention, a data 
recording medium for storing a program which makes a computer 
perform image signal decoding, and the program makes the 
computer perform differential decoding to the digital image 
signal by the image processing apparatus of eieim— 3r5"? ' '••y ' v 

By loading the program stored in the data recording /V/^-V^ 
medium thus constructed into the computer, differential 
decoding of the interlacing scanned image or the like, which 
is adapted to differential coding with no degradation of 
coding efficiency resulting from coding, is implemented by 
software. 
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Brief Description of the Drawings 

Figure 1 is a diagram showing a flow of image processing 
in a digital image padding method according to a first 
embodiment of the present invention. 

Figure 2 is a diagram showing resampling in the digita^l 
image padding method of the first embodiment. 

} 

Figures 3(a) and 3(b) are diagrams showing padding of 
pixel values in the digital image method of the first 
embodiment, and figure 3(c) is a diagram showing merging otf: 
pixels in the digital image padding method according to 
modification 1 of the first embodiment. J 

Figure 4 is a diagram showing a flow of image processing 
in a digital image padding method according to the 
modification 1 if the first embodiment of the present 
invention . 

Figure 5 is a diagram showing a flow of image processing 
in a digital image padding method according to a modification 
2 of the first embodiment of the present invention. 

Figure 6 is a diagram showing a flow of image processing 
in a digital image padding method according to a 
modification3 of the first embodiment of the present 
invention . 

Figure 7 is a diagram showing padding of pixels values 
in the digital image padding method according to the 
modification 3 of the first embodiment. 




Figures 8(a) and 8(b) are diagrams showing padding of 
pixels values in a digital image padding method according 
to a second embodiment of the present invention. 

Figure 9 is a block diagram for explaining an image 
coding apparatus according to a third embodiment of the* 
present invention, wherein figure 9(a) shows an entire 
structure of the image coding apparatus and figure 9(b) shows 
a structure of a padding section in the image coding 
apparatus. ' 

Figures 10 (a) -10 (d) are diagrams for explaining padding 
in the image coding apparatus of the third embodiment, ' 
wherein figure 10(a), 10(b), and 10(c) show rearrangement 
of pixels, frame padding, field padding, and merging for 
restoration . 

Figure 11 is a diagram showing a data structure of a 

coded image signal output from the image coding apparatus 
i 

of the third embodiment. 

Figure 12 is a block diagram showing an image coding 
apparatus according to a modification 1 of the third 
embodiment . 

Figure 13 is a block diagram for explaining an image 
coding apparatus according to a modification 2 of the third 
embodiment, wherein figure 13(a) shows an entire structure 
of the image coding apparatus and figure 13 (b) shows a padding 
section of the image coding apparatus. 
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Figure 14 is a block diagram showing an image decoding 
apparatus according to a fourth embodiment of the present 
invention . 

Figure 15 is a block diagram for explaining^ image 
decoding apparatus according to a modification 1 of the* 
fourth embodiment . 

I 

Figure 16 is a block diagram showing an image decoding 
apparatus according to a modification 2 of the fourth 
embodiment . f 

Figures 17(a), 17(b), and 17(c) are diagrams for 
explaining a data recording medium which contains a prograk 
for implementing image processing by the digital image I 
padding method or in the image processing apparatus, in a 
computer system . 

Figure 18 is a diagram showing replacement of pixel 
values in a prior art digital image padding method. 

Figure 19 is a diagram showing averaging of pixel values 
in the prior art digital image padding method. 

Figure 20 is a diagram showing -resampling of pixel 
values in the prior art digital image padding method. 

Oc ol MuJ u f o i CftiJ-i viucp Out — trh"£ Invent Ton 

Now, preferred embodiments of the present invention 
will be described with reference to figures 1-17. 
Embodiment 1 . 
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Figures 1-3 are diagrams for explaining a digital image 
padding method according to a first embodiment of the present 
invention. Figure 1 shows flow of processing of an image 
signal in the padding method. 

The digital image padding method of the first embodiment, 

as shown in figure 1, includes resampling an original image 

] 

space formed by a sampled input image signal to form plural 
small image spaces (step S12) and performing padding to each 
of the small image spaces by replacing sample values of' 
insignificant pixels with padding pixel values obtained by 
a prescribed method. I 

Figure 2 schematically shows resampling according tls 
the digital image padding method. An original image space 
301 corresponding to the input image signal, comprises 8 x • 
8 pixels, which are significant pixels having significant 
sample values and insignificant pixels having insignificant 
sample values. In figure 2, each pixel is represented by 
a square. The significant pixels are pixels (for example, 
pixels 303 and 304) of squares which are hatched, and the 
insignificant pixels are pixels (for example pixel 302) which 
are not hatched. The numbers of respective squares indicate 
the corresponding sample values. 

Small image spaces 401-404 resulting from the 
resampling each comprises 4x4 pixels. In this embodiment, 
the small image space 401 comprises pixels, which are 




positioned where odd-numbered pixel columns and odd-numbered 
pixel rows intersect in the original image space 301. The 
small image space 403 comprises pixels, which are positioned 
where even-numbered pixel columns and odd-numbered pixel 
rows intersect in the original image space 301. The small 
image space 402 comprises pixels, which are positioned wher^e 
odd-numbered pixel columns and even-numbered pixel rows 
intersect in the original image space 301. The small image 
space 404 comprises pixels, which are positioned where 
even-numbered pixel columns and even-numbered pixel rows 
intersect in the original image space 301. ^ 
Figures 3 schematically, shows padding in the digital 
image padding method of the first embodiment. The small 
image spaces 401 and 403 are padded as in the case of the 
prior art padding, to form a small image space 405 (see figure 
3(a)), and the small image spaces 402 and 404 are padded as 
in the case of the prior art padding, to form a small image 
space 406 (see figure 3(b)). 

Subsequently, operation and effects will be described. 
In the padding method of the first embodiment, in step 
Sll, a sampled digital image signal is input. In this 
embodiment, suppose that the digital image signal 
corresponds to an object, i.e., an image having an arbitrary 
shape, and forms the original image space 301 shown in figure 
2 including the object. 
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Note that the digital image signal may correspond to 

the interlacing scanned image (interlaced image) or the 

non-interlacing scanned image (progressive image) . Where 

the digital image signal corresponds to the progressive image, 

for example, an image of an object of a horizontal stripe 

pattern corresponds to the image in the original image space 

1 

301 xn figure 2. On the other hand, where the digital image 
signal corresponds to the interlaced image, an image which 
moves greatly corresponds to the image of the original imag*e 
space 301. In this interlaced image, as shown in figure 2, 
it sometimes looks like the object has a shape of two outlines . 

S^raes, the original image space 301 may be a regioln 
of the object (object region) itself comprising the object, 
or may be one of plural image spaces (blocks) each comprising 
a prescribed number of pixels (for example N x M samples), 
into which the object region is divided. In this embodiment, 
for convenience, assume that digital, image signals of 
object regions are sequentially input frame by frame in step 
Sll . 

In step S12, the original image space 301 of the digital 
image signal is resampled to generate plural small image 
space 401-404. Specifically, every other pixels are 
extracted in the horizontal direction and pixels on every 
other pixel row are extracted in the vertical direction in 
the original image space 301, to generate the four small image 
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spaces 401-404. 

These small image spaces will be described to be more 
detailed. Pixels at which odd-numbered pixel columns and 
odd-numbered pixel rows intersect in the original image space 
301 are extracted, to form the small image space 401, acid 

pixels at which even-numbered pixel columns and odd-numbered 

1 

pixel rows intersect are extracted, to form the small image 
space 403. Further, pixels at which odd-numbered pixel 
columns and even-numbered pixel rows intersect in the 
original image space 301 are extracted to form the small image 
space 402, and pixels at which even-numbered pixel column's 
and even-numbered pixel rows intersect in the original imagL 
space 301 are extracted to form the small image space 404. 

In subsequent step S13, each of the small image spaces 
401-404 are padded. Padding process of this embodiment 
comprises first to third steps S1-S3 in the prior art padding 
method (hereinafter referred to as first to third padding 
steps) shown in figures 18 and 19. 

In the first padding step, sample values of 
insignificant pixels are replaced wi.th sample values of 
significant pixels adjacent to the insignificant pixels in 
the horizontal direction in the order from inward to outward 
of the small image space, and then sample values of 
insignificant pixels are replaced with sample values of 
significant pixels or padded pixels (insignificant pixels 




whose sample values have been replaced) adjacent thereto in 

the vertical direction in the order from inward to outward 

of the small image space to which horizontal replacement 

process has been performed. 

In the second padding step, in the reversed order of 

the first padding step, sample values of insignificant pixels 

I 

are replaced with sample values of significant pixel adjacent 
to the insignificant pixels in the vertical direction in the 
order from inward to outward of the small image space, anil 
then sample values of insignificant pixels are replaced with 
sample values of significant pixels or padded pixels adjacent 
thereto in the horizontal direction in the order from inward 
to outward of the small image space to which vertical 
replacement process has been performed . 

In the third padding step, sample values of pixels in 
the small image space obtained from replacement process in 
the first padding step and sample values of pixels at the 
corresponding positions in the small image space obtained 
from the replacement process in the second padding step are 
averaged, to obtain sample values of pixels forming the small 
image space . 

As a result of this padding, the small image spaces 401 
and 403 are respectively changed into a small image space 
405 comprising pixels having sample values in figure 3(a) 
and the small image spaces 402 and 404 are respectively 
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changed into a small image space 406 comprising pixels having 
sample values in figure 3(b). 

Comparing these small image spaces resulting from 
padding to the odd-numbered field image 381 and the 
even-numbered field image space 382 (see figure 20) obtained 
by the prior art method, sample values of pixels in the sma^l 
image spaces 405. and 406 (see figure 3) are uniform, which 
shows that high-frequency components^ not introduced into 
the image signals of these small images. ' 

In final step S14, it is decided whether or not the 
resampled original image space corresponds to a final frame 
of the digital image signal. When decided that the originJl 
image space corresponds to the final frame, process in steps 
S11-S14 are completed (Step S15), whereas when decided that 
the original image space does not correspond to the final 
frame, t:he steps S11-S14 are performed to an original image 
space corresponding to a subsequent frame. 

Thus, in the first embodiment, every other pixels are 
extracted in the horizontal direction and pixels on every 
other pixel row are extracted in the vertical direction of 
the original image space 301 corresponding to the input image 
signal, and then the extracted plural pixels are rearranged 
to form 4 small image spaces 401-404, to which padding of 
pixel values are performed. Therefore, padding is 
performed to plural small image spaces where there is high 
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correlation of pixel values between adjacent scanning lines 

rather than an original image space where there is low 

correlation of pixel values between adjacent scanning lines. 

This results in padded small image spaces where there is high 

correlation of pixel values. As a result, it is possible 

to avoid introduction of high-frequency components into the 

) 

digital image signal of the interlaced image in process for 
padding the interlaced image which moves greatly, and hence 
it is possible to perform differential coding or differential 
decoding to the interlaced image or the like with high coding 
efficiency while suppressing degradation of coding ' 
efficiency resulting from padding. ! 

While in the first embodiment the original image space 
corresponds to the object region comprising the object, the 
original image space may correspond to one of blocks as coding 
units into which the object region is divided. In this case, 
in the step S14, it is decided whether or not the original 
image space corresponds to a final block of the object region. 
When decided that the original image space does not 
correspond to the final block, the steps S12-S14 are 
performed to an original image space corresponding to a 
subsequent block, whereas when decided that the original 
image space corresponds to the final block, it is decided 
whether or not the original image space corresponds to a final 
frame, and according to its decision result, it is decided 
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whether process is completed or the steps S12-S14 are 
performed to an original image space corresponding to a first 
block of a subsequent frame. 

In addition, while in the first embodiment the padding 
process comprises the first to third padding steps, only the 
first or second padding step may be performed for padding. 

Further, while in the first embodiment, as padding 
values for replacing sample values of insignificant pixels 
in the small image space, sample values of significant pixels 
adjacent to the insignificant pixels are used, an average 
of sample values of plural significant pixels in the small 
image space, or an average of sample values of plural ' 
significant pixels located at a boundary of an object in the 
small image space, may be used as the padding value. 

Still further, while in the first embodiment, every 
other pixels are extracted in the horizontal direction and 
pixels on every other line are extracted in the vertical 
direction of the original image space corresponding to the 
digital image space, in general, pixels may be extracted 
every (N + 1) -th (N: positive integer) pixel in the horizontal 
direction and every (N + l)-th pixel row in the vertical 
direction of the original image space. Intervals at which 
pixels are extracted may vary adaptively depending on a 
stripe pattern or the like of an image formed by a digital 
image signal. 
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Subsequently, modif ication* 1-3 of the first embodiment 
will be described. 

Modification 1 of First Embodiment . 

Figure 4 is a diagram for explaining a digital image 
padding method according to a modification 1 of the fir»st 
embodiment of the present invention and shows flow of ima^ge 
processing in the padding method. 

In this digital image padding method of the modification 
1 of the first embodiment, process for merging pixels in strip 
S16 is added to the steps S11-S15 in the digital padding method 
of the first embodiment in figure 1. * 

In step S16, all the pixels in the small image spacis 
405 and 406 which have been padded in step S13 are rearranged 
so that they have the same arrangement' as the original image 
space corresponding to the input digital image signal, to 
form a merged image space 407 which is as large as the original 
image space 301 (figure 3(c)). 

Specifically, pixels in the small image space 405 
obtained by padding the small image space 401 are disposed 
where odd-numbered pixel columns and odd-numbered pixel rows 
intersect in the merged image space which is twice as large 
as the small image space, and pixels in the small image space 
405 obtained by padding the small image space 403 are disposed 
where even-numbered pixel columns and odd-numbered pixel 
rows intersect in the merged image space. Further, pixels 
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in the small image space 406 obtained by padding the small 
image space 402 are disposed where odd-numbered pixel columns 
and even-numbered pixel rows intersect in the merged image 
space 407, and pixels in the small image space 406 obtained 
by padding the small image space 404 are disposed where 
even-numbered pixel columns and even-numbered pixel rows 
intersect in the merged image space. 

In this modification 1 thus constructed, every other 
pixels are extracted in the horizontal direction and pixels 
on every other pixel row are extracted in the vertical 
direction of the original image space 301 corresponding Jo 
the input digital image signal, and the extracted plural 
pixels are rearranged, to form the 4 small image spaces 
401-404, to which padding of pixel values is performed, and 
thereafter, pixels in the padded small image spaces 405 and 
406 are. rearranged, to form the merged image space 407 of 
the same pixel arrangement as the original image space 301. 
Therefore, image processing such as coding is performed to 
the padded merged space in the same manner that an unpadded 
original image space is processed without the necessity of 
changing processing of an image signal after padding 
depending on padding. 
Modification 2 of First Embodiment 

Figure 5 is a diagram showing a digital image padding 
method according to modification 2 of the first embodiment 
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of the present invention, and shows flow of image processing 
in the padding method. 

The digital image padding method of the modification 
2 of the first embodiment, as shown in figure 5, includes 
step S52 in which the degree of correlation of pixel values 
between adjacent pixel columns is evaluated for an original 
image space formed by a resampled input digital image signal, 
step S53 in which it is decided whether or not the original 
image space is resampled to form plural small image spacls 
according to the evaluation result, step S54 in which the 
original image space is resampled and then the small image 
space is padded, and step S55 in which the original imaje 
space is directly padded. 

Subsequently, operation and effect will be described. 
In the padding method of the modification 2 of the first 
embodiment, in step S51, a digital image signal forming a 
prescribed original image space where pixels are arranged 
m matrix is input. Subsequently in step S52, the degree 
of correlation of pixel values between adjacent pixel columns 
is computed for the original image space. More specifically, 
in this modification, differences of sample values of pixels 
between each pair of adjacent odd-numbered and even-numbered 
pixel columns in the original image space are computed, and 
absolute values of the differences are added for each pair 
of odd-numbered and even-numbered pixel columns, to 
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generate first evaluation values. 

In addition, differences (first differences) of sample 
values between each pair of adjacent odd-numbered pixel 
columns and differences (second differences) of sample 
values between each pair of adjacent even-numbered pixel 
columns are computed, and absolute values of these first and 
second differences are added for each pair of even-numbered 
pixel columns and for each pair of odd-numbered pixel columns, 
to generate second evaluation values. ) 

In step S53, comparison is made between the first and 
second evaluation values. From this comparison result, 1 
when the second evaluation values are smaller than the firlt 
evaluation values, that is, correlation of pixel values 
between the odd-numbered and even-numbered pixel columns is 
lower than correlation of pixel values between odd-numbered 
pixel columns and between even-numbered pixel columns, in 
step S54, the original image space is resampled, and then 
padded. Specifically, pixels on every other pixel column 
are extracted in the original image space, to generate a first 
small image space comprising pixels forming odd-numbered 
pixel columns and a second small image space comprising 
pixels forming even-numbered pixel columns, and then padding 
of pixels is performed to each small image space. 

On the other hand, when the second evaluation values 
are not smaller than the first evaluation values from the 
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comparison result in step S53, in step S55, the original image 
space is padded without resampling. The padding herein is 
identical to padding in step S13 of the first embodiment. 

Thus, in the modification 2 of the first embodiment, 
correlation of pixel values between adjacent pixel columns 
is decided for the original image space corresponding to the 
input image signal, and according to the decision result, 
padding is performed to the small image space in which the 
original image space has been resampled or is directly 
performed to the original image space. Therefore, it is 
possible to perform padding to an input digital image signal 
according to its quality, and to avoid performing paddirlg 
to the original image space where there is low correlation 
of pixel rows between adjacent pixel columns. As a result, 
padding in coding or decoding process bf an image signal can 
be performed without degrading coding efficiency. 

While in this modification 2 of the first embodiment, 
correlation of pixel values between pixel columns is decided, 
and the original image space is divided into the small image 
space composed of odd-numbered pixel columns and a small 
image space composed of even-numbered pixel columns, the 
correlation of pixel values may be decided between pixel rows 
in the original image space. Also in this case, as in the 
modification 2 of the first embodiment, two evaluation values 
are computed and comparison is made between them to make 
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decision, and then according to the decision result, it is 
decided whether or not resampling is performed to the 
original image space in the vertical direction prior to 
padding, that is, it is decided whether or not the original 
image space is divided into the small image space (odd*- 
numbered field) composed of odd-numbered pixel rows and Ijhe 
small image space (even-numbered field) composed of 
even-numbered pixel rows. 

Besides, while in this modification 2 of the first 
embodiment, pixels on every other pixel column (or every 
other pixel row) are extracted in the original image space, 
and comparison is made for correlation of sample value! 
between extracted pixel columns (or pixel rows) , pixels may 
be extracted every (N + l)-th (N=l, 2, 3, ...) pixel in the 
vertical (or horizontal) direction of the original image 
space, correlation between the extracted pixels may be 
computed to generate plural evaluation values, and according 
to largeness of these evaluation values, it may be decided 
whether or not resampling is performed every (N + l)-th 
pixel in the vertical (or horizontal) direction prior to 
padding . 

Further, a function used in computing evaluation values 
from sample values of pixels extracted in the original image 
space, is not limited to a function in which sum of absolute ' 
values of differences between sample values is used as 
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evaluation values, and the other functions may be employed. 
Moreover, the evaluation values may 'be computed only from 
significant samples. 

Modification 3 of First Embodiment 

Figure 6 is a diagram showing a digital image padding 
method according to a modification 3 of the first embodiment 
and shows flow of image processing in the padding method. 

The digital image padding method of the modification 
3 of the first embodiment, as shows in figure 6, includes 
step S62 in which the degree of correlation of pixel values 
between adjacent scanning lines (pixel rows) is evaluated 
for an original image space formed by a sampled input digital 
image signal, step S63 in which the degree of correlation 
of pixel values in fields of the original image space is 
decided, step S64 in which the original image space is 
resampled to form a small image space corresponding to an 
odd-numbered field and a small image space corresponding to 
an even-numbered field, and padding is performed for each 
small image space, i.e., field by field, and a step S65 in 
which the original image space is padded frame by frame. 

Subsequently, operation and effects will be described. 
In the padding method of the modification 3 of the first 
embodiment, in step S61, a digital image signal forming a 

prescribed original image space in which pixels are arranged 
• ... 

m matrxx is input. In this modification 3, suppose that 
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the digital image signal corresponds to the interlaced image, 
and forms the image space 301 shown in figure 2. 

Subsequently in step S62, correlation of pixel values 
between scanning lines in the original image space is 
computed. Specifically, by the method described by means 
of figure 5, first evaluation values indicating correlation 
of pixel values between odd-numbered and even-numbered lines 
in the original image space are generated, and second 
evaluation values indicating correlation of pixel values 
between odd-numbered lines and between even-numbered lines 
in the original image space are generated. J 

Then in step S63, comparison is made between the firlt 
and second evaluation values, to decide whether or not 
correlation of pixel values in fields is higher than 
correlation of pixel values in a frame. 

When decided that the second evaluation values are 
smaller-than the first values, that is, correlation of pixel 
values in the fields is higher than correlation of pixel 
values in the frame, in step S64, the original image space 
formed by the digital image signal is padded field by field. 
In other words, a small image space composed of odd-numbered 
lines and a small image space composed of even-numbered lines 
are generated, and then padding is performed for each small 
image space (field) . On the other hand, when decided that 
the second evaluation values are not smaller than the first 
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values, in step S65, the original image space is padded frame 
by frame, that is, with the odd-numbered lines and the 
even-numbered lines merged. 

Figure 7 is a diagram for explaining field by field 
padding of the original image space 301 in figure 2. Figure 
7 (a) shows a small image space 701 corresponding to the, 
odd-numbered field in the original image space 301, and a 
small jjr^aJrge space 703 obtained by padding the small image 
space 701. Figure 7(b) shows a small image space 702 ' 
corresponding to an even-numbered field in the original image 



space 301, and shows a small image space 704 obtained by 
padding the small image space 702. ' 

As can be seen from figure 7, the small image spaces 
701 and 702 can be padded without introducing high-frequency 
components into them. 

In this modification 3 of the first embodiment, it is 
decided -whether the original image space corresponding to 
the input image digital image signal is padded field by field 
or frame by frame, and padding of pixel values is performed 
to the original image space in one of these units. Therefore, 
it is possible to perform padding to the interlaced image 
without degrading efficiency in coding of the image signal 
of to the interlaced image. 
Embodiment 2 . 

Figure 8 is a diagram for explaining a digital image 
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padding method according to a second embodiment of the 
present invention, and shows an original image space formed 
by a digital image signal to be padded, and a padded image 
space in which the original image space has been padded. 

In the digital image padding method of the second 
embodiment, the step S12 for resampling and step S13 for 
padding by the digital image padding method of the first 
embodiment, are replaced by a step in which an original image 
space 801 corresponding to a digital image signal is directly 
padded, that is, sample values of insignificant pixels are 
replaced with padding values obtained from sample values 'of 
significant pixels according to a prescribed rule. In this 
embodiment, used as the significant pixels used in generating 
the padding values are, pixels which are adjacent to 
insignificant pixels in the horizontal direction and pixels 
which are adjacent thereto one pixel spaced apart therefrom 
in the vertical direction of the original image space. 

Specifically, the digital image padding of the second 
embodiment includes a first replacement step S21 (see figure 
8(a)) for sequentially replacing sample values of 
insignificant pixels in the horizontal direction and a second 
replacement step S22 (see figure 8(b)) for replacing sample 
values of insignificant pixels in the vertical direction. 
In the second replacement step S22, sample values of 
insignificant pixels in an image space 801a obtained through 




the first replacement step S>zi , are -replaced with sample 
values of pixels on a pixel row adjacent to a pixel row 
comprising the insignificant pixels, one pixel row spaced 
apart therefrom. 

Subsequently, operation and effects will be described. 
Since there is low correlation of pixel values between 
an odd-numbered line (pixel row) and an even-numbered line 
(pixel row) in the original image space 801, effectiveness 
is provided by the method of dividing the original image sp'ace 
801 into the small image space composed of odd-numbered lines 
and the small image space composed of even-numbered lin'es, 
and padding respective small image spaces, as already! 
described in the first embodiment. On the other hand, for 
an image signal forming an original image space with high 
correlation between odd-numbered lines and between even- 
numbered lines, there is another effective method, in which 
sample values to be padded, i.e., insignificant pixels in 
the original image space, are replaced with sample values 
of significant pixels located one pixel spaced apart 
therefrom in the vertical direction (one pixel is located 
adjacently to the other pixel with one pixel interposed 
between them in the vertical direction) . 

So, in the digital padding method of the second 
embodiment, as shown in figure 8(a), sample values of 
insignificant pixels are replaced with sample values of 
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significant pixels adjacent thereto, for insignificant 
pixels aligned in the horizontal direction, in the order 
inward to outward of the original image space 801 (first 
replacement step) . 

Subsequently, as shown in figure 8(b),- sample values 
of insignificant pixels on a pixel row 812 are replaced with 
sample values of pixels on a pixel row 814 adjacent thereto 
with a pixel row 813 interposed between them in the image 
space 801a obtained through the first replacement step sA, 
and sample values of insignificant pixels on a pixel row 811 
are replaced with sample values of pixels on the pixel row 
813 adjacent thereto with the pixel row 812 interposed! 
between them in the image space 801a (second replacement step 
S22) . 

In the digital image padding method of the second 
embodiment, the same effects as provided by the digital image 
padding -method of the first embodiment in which the original 
image space corresponding to the input image signal is 
resampled, and the resulting image space is padded, are 
obtained. Specifically, even for the case of the digital 
image signal corresponding to the interlaced image, it is 
possible to avoid introduction of high-frequency components 
into the digital image signal of the original image space 
in padding process, and hence, differential coding or 
differential decoding of the interlaced image or the like 
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can be performed with high efficiency while suppressing 
degradation of coding efficiency resulting from padding. 

While in the second embodiment, the sample values of 
the significant pixels located one sample spaced apart from 
the insignificant pixels in the vertical direction are used 
as the padding pixel values in the replacement step S22, t^he 
sample values for the padding values are not limited thereto, 
and sample values of significant pixels located N samples 
spaced apart from the insignificant pixels in the vertical 
direction may be generally used in padding of sample values 
of the insignificant pixels. 1 

In addition, while in the second embodiment, sample 
values of insignificant pixels are sequentially replaced 
first in the horizontal direction, and then every other 
sample values of insignificant pixels are replaced in the 
vertical direction, sample values of insignificant pixels 
may be sequentially replaced first in the vertical direction 
and then every other sample values of insignificant pixels 
are replaced in the horizontal direction. in this case, it 
is possible to effectively perform padding to a progressive 
image having a vertical stripe pattern. 

Hereinafter, description is given of an image coding 
apparatus using the digital image padding method of the 
present invention in a third embodiment and an image decoding 
apparatus using the digital image padding method in a fourth 
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embodiment . 
Embodiment 3. 

Figure 9 is a block diagram showing the image coding 
apparatus as an image processing apparatus according to the 
third embodiment of the present invention, wherein figure 
9(a) shows an entire construction of the image coding 
apparatus and figure 9 (b) shows a padding section of the image 
coding apparatus. 

An image coding apparatus 100a of the third embodiment 
includes a blocking unit 101 for dividing an input digital 
image signal Sg supplied to an input terminal 101a into 
signals corresponding to blocks each comprising 16 x ll 
pixels as coding units and outputting the resulting image 
signal (hereinafter referred to as a blocked image signal) 
Bg, an interlace detecting unit 104 for distinguishing 
between the interlaced image and the progressive image on 
the basis of an output Bg of the blocking unit 101, for an 
image corresponding to the digital image Sg, and outputting 
a sample identifier Di according to the result, and a 
predicting section 120a for producing a prediction image Prg 
of a target block to-be-coded. 

The image coding apparatus 100a further includes a 
subtracter 102 for outputting difference between the blocked 
image signal Bg of the target block and the prediction signal • 

Prg as a difference signal (prediction error signal) Dg, data 

A 



55 



compressing rn^hs 110 for performing information compression 
to the output Dg of the subtracter 102 according to the sample 
identifier Di and outputting a compressed difference signal 
Qg, and a variable length encoder (VLC) 103 for performing 
variable length coding to the output Qg of the data 
compressxng rr^ns 110 and outputting a coded image signal 
Eg. 1 

The data compressing m^erTs 110 includes a pixel 
rearrangement unit 113 for rearranging pixels of a difference 
image signal formed by the difference signal Dg according 
to the sample identifier Di, a DCT unit 111 for performing 
discrete cosine transform to an output PDg of the pixel! 
rearrangement unit 113, and a quantizer 112 for quantizing 
an output Tg of the DCT unit 111 and outputting quantization 
coefficients as the compressed difference signal Qg . The 
discrete cosine transform is performed for each of 4 blocks 
each comprising 8 x 8 pixels into which the block comprising 
16 x 16 pixels is divided. Where the rearrangement has been 
performed, a block 250c is divided into small blocks 251-254 
as shown in figure 10(c), whereas where the rearrangement 
has not been performed, a block 250b is divided into small 
blocks 255-258 as shown in figure 10(b). 

The pixel rearrangement unit 113 is used for rearranging 
pixels in the image space 250 corresponding to the difference . 
signal Dg to generate a rearranged image space 250a according 
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to the sample identifier Di as shown in figure 10(a). in 
this rearranged image space 250a, an upper region 250al 
comprising 16 x 8 pixels comprises pixels on odd-numbered 
pixel rows and a lower region 250a2 comprising 16x8 pixels 
comprises pixels on even-numbered pixel rows in the ima,ge 
space 250. 

aS&s^ P redi cting section 120a includes K^ta expanding 

m^ams 130 for performing expansion to the output Qg of the 

is/ (JL~- 

data compressing m^rls 110 and outputting an expanded I 
difference signal Itg, and an adder 123 for adding the 
expanded difference signal Itg to the prediction image signal 
Prg and outputting a restored image signal Lrg. The dala 
expanding m^ams 130 includes an inverse quantizer 121 for 
inversely quantizing the output Qg of the data compressing 
jneefis 110, and an IDCT unit 122 for performing inverse DCT 
to an output IQg of the inverse quantizer 121 by transforming 
data in-a frequency domain into data in a spatial domain, 
and outputting the expanded difference signal Itg. 

t&y/aZ^ PrediCting section 12 0* further includes V p^dding 
mgums 150 for per forming padding to the restored image signal 

Lrg fr °™JJ he adder 123 according to the sample identifier 
Di, and prediction signal producing i^fe's^ 0 for producing 
the prediction image signal Prg from a padded restored image 
signal Pag. 

The prediction signal producing p«rans 140 includes a 
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f ^^/^° ry St ° ring th€ ° Utput Pa< 3 of padding 

rn^HTs 150, a motion detecting unit (ME) 125 for computing 
a motion vector MV of the target block based on the image 
data Mg stored in the frame memory 124, the blocked image 
signal Bg , and the sample identifier Di and outputting the 
MV, and a motion compensating unit (MC) 126 for generating 
an address Addl of the frame memory 124 according to the motion 
vector MV from the motion detecting unit 125 and outputting 
an image signal in a memory area of the address Addl as t'he 
prediction signal Prg. The motion detecting unit 125 is used 
for generating a motion vector of a frame and a motion vector 
of a field according to the sample identifier Di . The motiln 
vector MV and the sample identifier Di are variable length 
coded by the variable length encoder 103 together with the 
compressed difference signal Qg . 

As shown in figure 9(b), the padding^^?^? 0 includes 
a frame -padding unit 128a for padding the output Lrg of the 
adder 123 frame by frame, a field padding unit 128b for padding 
the restored image signal Lrg from the adder 123 field by 
field, a primary switch 127a provided at a primary stage of 
these padding units, for supplying one of the padding units 
with the restored image signal Lrg from the adder 123, a 
secondary switch 127b provided at a next stage of these 
padding units, for selecting one of outputs Pal and Pa2 of ■ 
respective padding units according to the sample identifier 
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Di, and a pixel rearrangement unit 129 for receiving a padded 
restored signal as a selected signal Spa from the secondary 
switch 127b and rearranging pixels in an image space formed 
by the padded restored image signal. 

As shown in figure 10(b), the frame padding unit 128a 
is used for performing padding the restored image signal ijrg 
of 4 small blocks (8 x 8 pixels) 255-258 in an image space 
250b (16 x 16 pixels) corresponding to the restored image 
signal at a time . As shown in figure 10 (c) , the field paddi'ng 
unit 128b is used for performing padding to the restored image 
signal Lrg of small blocks 251 and 252 composed of first fie'ld 
lines in an upper region 250cl of an image space 250c of tie 
restored image signal at a time, and padding the signal Lrg 
of small blocks 253 and 254 composed of second field lines 
in a lower region 250c2 of the image space 250c at a time. 
The pixel rearrangement unit 129 is used for rearranging 
pixels in an image space 250d corresponding to a padded 
restored image signal from the field padding unit 128b and 
outputting the image signal Pag corresponding to a merged 
restored image space 250e where 8 pixel rows in the upper 
region 250dl and 8 pixel rows in the lower region 250d2 of 
the image space 250d are alternately arranged 

Next, data structure of a coded image signal output from 
the image coding apparatus 100a will now be described. 
Figure 11 schematically shows a data structure of the coded 
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image signal (bit stream) . 

In this bit stream 200, a bit string F corresponding 
to a frame image comprises a synchronous image signal 203 
indicative of a starting point of a code string, a 
quantization step value 204 indicative of a quantization 
scale (quantization step) for use by quantization in coding 

process, and data D(l), 0(2), (D3) D (i), corresponding 

to blocks (macroblocks) MB(1), MB(2), MB(3) MB ( i ) each 

comprising 16 x 16 pixels. Reference numerals 205 , 208, afid 
211 designate sample identifiers each indica^ii^he ther or 
not correlation between pixels in fields is higher than 1 
correlation between pixel values in a frame. Reference' 
numerals 206, 209, and 212 designate variable length coded 
motion vectors. Reference numerals 207, 210, and 213 
designate DCT coefficients of subblocks each comprising 8 
x 8 pixels of each block. The data D(l), data D (2), and 
D(3) corresponding to respective macro blocks comprises, the 
sample identifier 205, the motion vector 206, and the DCT 
coefficient 207, the sample identifier 208 , the motion vector 
209, and the DCT coefficient 210, and the sample identifier 
211, the motion vector 212, and the DCT coefficient 213, 
respectively. In figure 11, reference numeral 214 
designates a synchronous image signal of a frame subsequent 
to the frame F. 

In this embodiment, the synchronous image signals 203 
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and 214 are respectively disposed at the head of the 
corresponding bit streams of frame images, and following the 
synchronous image signal 203, aligned are the quantization 
step value 204 for use by quantization, the sample identifier 
205, the motion vector 206, and the DCT coefficient 207 ,as 
compressed data. The synchronous image signal 203 
indicates the head of the compressed data of the image 200, 
and is represented by unique 32 bits. The quantization step 
204 is a parameter of 5 bits used for inversely quantizing 
the DCT coefficient. 

Subsequently, operation will be described. J 
When the digital' image signal Sg is input to the image 
coding apparatus 100a, the blocking unit 101 performs 
blocking so that the digital image signal Sg is divided into 
signals corresponding to image spaces (blocks) each 
comprising 16 x 16 pixels into which one frame is divided, 
to be output. The subtracter 102 computes difference 
between the blocked image signal Bg and the prediction image 
signal Prg from the predicting section 120a and outputs the 
resulting difference signal (prediction image signal) Dg to 
the data compressing means 110. At this time, the interlace 
detecting unit 104 identifies an interlaced image based on 
the image signal of each block (blocked image signal) Bg, for 
the input image, and outputs the sample identifier Di 
according to the decision result. 



When the difference signal Dg is supplied to the data 
compressing rne^s 110, the pixel rearrangement unit 113 
subjects the difference signal Dg to rearrangement according 
to the sample identifier Di. Specifically, where the 
digital image signal corresponds to the interlaced image, 
the difference signal Dg is subjected to rearrangement ^nd 
input to the DCT unit 111, whereas where the digital image 
signal corresponds to the progressive image, the difference 
signal Dg is directly input to the DCT unit 111 without' 
rearrangement. The DCT unit 111 performs DCT to the output 
PDg of the pixel rearrangement unit 113. More specifically, 
for the block corresponding to the interlaced image, th'e 
block is divided into subblock as shown in figure 10(c), and 
for the block corresponding to the progressive image, the 
block is divided into subblocks as shown in figure 10(b), 
and then the output PDg is subjected to DCT for each subblock. 
The quantizer 112 quantizes an output Tg of the DCT unit 111 
by a quantization step and outputs quantization coefficients 
Qg of the difference signal to the -variable length encoder 
103 and the information expanding section 130. 

The information expanding section 130 expands and 
converts the compressed difference signal Qg of the target 
block into the expanded difference signal ITg. More 
specifically, in this embodiment, the inverse quantizer 121 
inversely quantizes the compressed difference signal Qg, and 




further, the inverse discrete cosine transform (IDCT) unit 

122 subjects the output IQg of the quantizer 121 to IDCT by 

transforming a signal in a frequency domain into a signal 

into a signal in a spatial domain. 

The adder 123 adds the expanded difference signal ITg 

to the prediction image signal Prg and outputs the restored 

) 

image signal Lrg to the padding means 150. 

In the padding qae^rfts 150, the restored image signal Lrg 
is supplied to one of the frame padding unit 128a and trie 
field padding unit 128b through the switch 127a. At this 
time, switching of the switch 127a is controlled according 
to the sample identifier Di . Where a value of the sampJe 
identifier Di is 1, that is, there is high correlation between 
pixels in the fields, the restored image signal Lrg is 
supplied to the field padding unit 128b through the switch 
127a and is subjected to intra-field padding therein. 
Specifically, as shown in figure 10 (c) , padding is performed 
to the restored image signals of small blocks 251 and 252 
composed of lines of the first field of the restored image 
space 250c at a time, and the restored image signals of small 
blocks 253 and 254 composed of lines of the second field of 
the restored image space 250c at a time. 

On the other hand, where the value of the sample 
identifier Di is 0, that is, where there is high correlation 
between pixel values in the frame, the restored image signal 



63 



Lrg is input to the frame padding unit 128a through the switch 
127a and is subjected to intra-frame padding therein. 
Specifically, as shown in figure 10 (b) , padding is performed 
to the small blocks 255-258 of the restored image space 250b 
at a time. 



Thus padded restored image signals Pal and Pa2 are sent 
to the pixel rearrangement unit 129 through the primary 
switch 127b. The pixel rearrangement unit 129 rearranges 
Pixels of the first field and pixels of the second field do 
that they have pixel rearrangement of the interlaced image 
according to the sample identifier Di, for the macrobloc'k 
250d which has been padded field. by field. Specif ical 1 J, 
Pixels in the restored image space 250d in figure 10(d) are 
rearranged so that pixel rows in the upper region 250dl and 
pixel rows in the lower region 250d2 are alternately disposed, 
resulting in an image signal forming an image space 250e of 
the same pixel arrangement as an image space formed by the 
difference signal Dg. On the other hand, for the block which 
has been padded frame by frame, pixels are not rearranged 
by the pixel rearrangement unit 129. 

The padded restored image signal Pag which has been thus 
rearranged is stored in the frame memory 124 of the prediction 
signal producing section 140. 

In the prediction signal producing section 140, the 
image signal Bg of the target block is input to the motion 
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detection unit 125, and simultaneously, the padded restored 
image signal Pag is read from the frame memory 124 onto the 
motion detecting unit 125 as an image signal of a coded 
previous frame. 

The motion detecting unit 125 detects a region of the 
previous frame including an image signal which has the 
smallest error between the image signal Bg of the target block 
and the same as a prediction region and, outputs motion 
displacement information (motion vector) MV indicating "fa 
position of the prediction region, by block matching or the 
like. The motion vector MV is sent to the motion i 
compensating unit 126, which generates an address Addl f>f 
the frame memory 124 from the motion vector, and reads an 
image signal corresponding to the prediction region of the 
previous frame using the address Addl as the prediction image 
signal Prg. 

Further, the motion vector MV of the target block, the 
sample identifier Di, and the coded difference signal Qg are 
supplied to the variable length encoder 103, where they are 
converted into the corresponding to variable length codes, 
and output from an output terminal 101b as the coded image 
signal Eg. 

Thus, in the third embodiment, the predicting section 
120b performs padding depending upon whether the digital 
image signal corresponds to the interlaced image or the 
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progressive image. Therefore, it is possible to perform 
differential coding to the interlaced image with high 
efficiency . 

Subsequently, modifications of the third embodiment 
will be described. * 
Modification 1 of Third Embodiment ^ 

Figure 12 is a diagram showing an image coding apparatus 
according to a modification 1 of the third embodiment. 

In an image coding apparatus 100b of the modification 

1 of the third embodiment, only a predicting section 120b 

I 

differs from that of the image coding -apparatus 100a of the 
third embodiment, and in the predicting section 120b of the 



image coding apparatus 100b, the padding Jm^a^s 150 of the 
predicting section 120a of the first embodiment is replaced 
with a merging unit 151 which performs merging wherein pixels 
in an image space formed by the restored image signal Lrg 
from the adder 123 are rearranged, and^padding mefns 150b 
which performs padding to an output of the merging unit 151. 
■ i The merging unit 151 is, as shown in figure 10 (d) , used 

for rearranging pixels by alternately arranging pixels rows 
in the upper region 250dl and pixel rows in the lower region 
250d2 of the image space 250d. The padding poe^rfis 150b 
includes the primary and secondary switches 127a and 127b, 
the frame padding unit 128a, and the field padding unit 128b 
of the padding ^^ns'^150 of the third embodiment, the field 
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padding unit 128b performing padding by the digital image 
padding method of the second embodiment. The other 
components are identical to those of the third embodiment. 

In the modification 1 of the third embodiment thus 
constructed, the same effects as provided by the third t 
embodiment are obtained. 

Modification 2 of Third Embodiment ' 
<5* Figure 13 is a diagram showing an image coding apparatus 

according to a modification of the third embodiment of the! 

present invention, wherein figure 13(a) shows an entire 

construction of^he^ image coding apparatus and figure 13(b)! 

shows^padding means' 150c of the data compressing ^^hsflfoJ 

of the image coding apparatus. 

l-f- In an image coding apparatus 100c of the modification 

2 of the third embodiment, only data compressing ^ekhs 100c 

differs from that of the image coding apparatus 100a of the 

third embodiment, and the data compressing jjeaws' 110c of the 

image coding apparatus 100c is provided wit h^padding r^iSfs^ 

150c which performs padding to an output PDg of the pixel 

rearrangement unit 113 between the pixel rearrangement unit 

113 and the DCT unit 111 at the next stage of the data 
cfet/ic£- 

compressing rjLswrris 110 of the third embodiment . 

p (£0i/t£-&- 

— cj» The padding me^rrs 150c includes a frame padding unit 
158a for performing padding to the output PDg of the 
rearrangement unit 113 frame by frame, a field padding unit 
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158b for performing padding to the output PDg of the 
rearrangement unit 113 field by field, a primary switch 157a 
provided at a previous stage of these padding units, for 
supplying one of these padding units with the output PDg 
according to the sample identifier Di, and a secondary switch 
157b provided at a next stage of these padding units, for 
selecting one of outputs PDgl and PDg2 of the padding units 
according to the sample identifier Di . Constructions of the 
frame padding unit 158a and the field padding unit 158b ate 
identical to those of the frame padding unit 128a and the 
field padding unit 128b of the third embodiment. ' 

In this modification 2 of the third embodiment, sinie 
padded difference signal is supplied to the DCT unit 111, 
efficiency in DCD process is increased, in addition to the 
effects provided by the third embodiment. 
Embodiment 4 . 

Figure 14 is a block diagram showing an image decoding 
apparatus according to a fourth embodiment of the present 
invention . 

An image decoding apparatus 900a of the fourth 
embodiment is used to receive a bit stream 200 of the data 
structure in figure 11 as a coded image signal Eg in which 
an image signal has been compressively coded and perform 
decoding to the coded image signal Eg. 

Specifically, the image decoding apparatus 900a 
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includes a data analyzing unit 902 for receiving a coded 
difference signal Eg of a target block to be decoded (target 
block) , and performing analysis and variable length decoding 
to the coded difference signal Eg/data expanding joa^ms 903 
for expanding an output AEg of the data analyzing unit 902 
and outputting an expanded image signal DEg, an adder 906 
for adding the output DEg to a prediction image signal Pg2^ 
of the target block, an inverse blocking unit 907 for 
integrating reproduced image signals, of respective blocks') 
as an output PEg of the adder 906 and outputting an image 
output signal REg of a scanning line structure, and a ] 
predicting section 910a for producing the prediction imagef 
signal Pg2 . 

]- The data expanding pe^ns 903 comprises an inverse 

quantizer 904 for inversely quantizing the output AEg of the 
data analyzing unit 902, and an IDCT unit 905 for performing 
IDCT (inverse discrete cosine transform) as one type of 
inverse frequency transform, to the output DEgl of the 
inverse quantizer 904, and outputting the expanded 
difference image signal DEg. 
</ The predicting section 910a includes padding mp-erns 920a 

for performing padding to the output PEg of the adder 906, 
a frame memory 909 which contains an output Pa of the padding 
jjie-alTs 920a as a reproduced image signal, and a motion 
compensating unit 911 for generating an address Add2 of the 



U 
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frame memory 909 according to an output Rg2 of the £raM memory 
909 and a motion vector MV of the block which has been decoded 
by the data analyzing unit 902, and reading a prediction image 
signal Pg 2 of the target block from the frame memory 909 which 
contains *«°*<»«"^taR92 using the address Add2. 

The padding ^ s 920a includes a frame padding 
921a for performing padding to the output PEg of the adder 
906 frame by frame, a field padding unit 921b for performing 
padding to the output PEg field by field, a primary switdh 
922 provided at a previous stage of these padding units, for 
supplying one of them with the output PEg according to th'e 
sample identifier Di from the data analyzing unit 902, i 
secondary switch 923 provided at a next stage thereof, for 
selecting one of outputs Pa! and Pa2 of respective padding 
units according to the sample identifier, and a pixel 
rearrangement unit 924 for rearranging pixels of the padded 
image signal as a selected signa! Pas from the secondary 
switch 923. 

Besides, reference numerals 901a and 901b designate 
input and output terminals of the image decoding apparatus 

900a, respectively. 

Subsequently, operation will be described. 

When the compressive^ coded image signal Eg, for 
example, the bit stream 200 in figure 11 is input to the input . 
terminal 901a, the data analyzing unit 902 performs variable 
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length decoding to the coded image signal by analysis. The 



resulting decoded data is output to the data expanding j^ii-erfis 
903 as a compressed difference signal AEg in which a 
quantization step or a DCT coefficient has been compressed. 
The data analyzing unit 902 also outputs the motion vector 
MV and the sample identifier Di to the predicting section 

910a. . , 

The data expanding gjjMrfis 903 expands the compressed 

I 

difference signal to be restored to an expanded difference 
signal of the target block. Specifically, the inverse 
quantizer 904 inversely quantizes the compressed dif ferenc^ 
signal, and then the IDCT unit 905 performs inverse frequency 
transform to the output DEgl of the quantizer 904, by 
transforming a signal in a frequency domain into a signal 
in a spatial domain. 

The motion compensating unit 911 generates the address 
Add2 for accessing the frame memory 909 according to motion 
vector MV from the data analyzing unit 902 and extracts the 
prediction image signal Pg2 of the target block from the 
reproduced image signal stored in the frame memory 909. The 
adder 906 adds the output DEg of the data expanding ra^arns 
to the prediction image signal Pg2 and outputs the reproduced 
image signal PEg of the target block. 

The reproduced image signal is supplied to the inverse 
blocking unit 907 and the predicting section 910a. 
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Th. «o= k in 9 unit 907 integrates th „ reproduced 

*age -gnals to convert int<> ^ 

«gn.l of the scanning line structure. 

In the predicting section 910a , k „ 
.. ' the re Produced image 

signal PEg is supplied to one of the f 

o one of the frame padding unit 921a 
and the field padding unit 921b through ,k 

„ . through the primary switch 

Switching of the primary switch 922 is controlled in 
accordance with the sample identifier oi from the data 
analy 2 i„ g unit 90 , ^ ^ ^ ^ ^ ^ 

•n t " l ;. that ls - there is hi9h co - ei — — 

to th f r£Pr0dUCed Si - al - — ^ 

the field padding unit 921b through tfc . switch J 

« -ubjecfd to intra-fieid padding therein 

specifically, as shown in fi gure 10(c) . padding 
-formed to the reproduced image signal P Eg of small bloc.s 
251 and 252 composed of first field lines in th. ■ 
2"c at 'a time and the ■ 

" d S19nal ^ ° f —11 blocs 253 and 

254 composed of second field lines in f h ■ 

o -lines in the image space 250c 

corresponding to the reproduced image signal P Eg at a time 
on the other hand, where the value of the sample 

; ;;; lfi ; r is °- that — - — - elation between 

P».l values in the frame, the reproduced image signal P Eg 
3 supplied to the frame padding unit 921a through the switch 
and is subjected to intra-frame padding therein 
specifically, as shown in fi gur e 10 <b, , padding lm pe rformed 
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to the reproduced image signal PEg of the small blocks 255-258 
in the image space 250b corresponding to the reproduced image 
signal PEg at a time. 

Thus padded reproduced image signal Pas is sent to the 
pixel rearrangement unit 924 through the secondary switch 

923. The pixel rearrangement unit 924 rearranges pixels of 

I 

the first field and pixels of the second field so that they 
have pixel rearrangement of the interlaced image, for the 
block which has been padded field by field. For example, 
the pixel rearrangement unit 924 converts the reproduced 
image signal Pas into a reproduced image signal of the imige 
space 250e where lines of the upper region 250dl in the imige 
space 250d and lines of the lower region 250d2 in the small 
image space 250d (see figure 10 (d) ) are alternately arranged. 
On the other hand, for the block which has been padded frame 
by frame, the pixel rearrangement unit 924 does not rearrange 
pixels.- Thus rearranged reproduced image signal Pa is 
stored in the frame memory 909. 

In this fourth embodiment, described is a case where 
the reproduced image signal PEg corresponding to the 
interlaced image forms the image space. 250d in figure 10 (d) , 
the upper and lower regions 250dl and 250d2 of which are 
composed of odd-numbered lines and even-numbered lines, 
respectively. On the other hand, for the case where the 
reproduced image signal PEg is output with the fields 1 and 
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2 merged, that is, the reproduced image signal PEg 
corresponding to the interlaced image forms the image space 
250 in figure 10(a) where odd-numbered and even-numbered 
lines are alternately arranged, it becomes necessary to 
change an implementing method of the field padding unit 921b. 
In this case, prior to padding, pixels of the reproduced image 
signal PEg must be rearranged, to be converted into the image 
space 250d whose upper and lower regions are fields 1 and 
2, respectively as shown in figure 10(d). 

Also, in the case where the reproduced image signal PEg 
is output with the fields 1 and 2 merged as described above, 
sample values of pixels to-be-padded (insignificant pixels) 
may be replaced by sample values of significant pixels one 
sample spaced apart from the insignificant pixels without 
pixel rearrangement, for the image signal corresponding to 
a frame, rather than the field padding unit 921b has 
rearranged pixels so that odd-numbered lines and even- 
numbered lines of the frame are collected into the upper and 
lower regions, respectively, followed by padding field by 
field. 

Besides, while in the fourth embodiment, pixels of the 
output of the field padding unit are rearranged, the pixel 
rearrangement unit 924 may be dispensed with when the output 
of the field padding unit is directly stored in the frame 
memory, because there is no necessity of rearranging pixels. 



In addition, while in the fourth embodiment, the coded 
image signal which has been compressed by DCT is input, the 
input coded image signal may be a coded image signal which 
has been coded by wavelet, fractal, and the like. Also in 
this case, as in the case of the fourth embodiment, switching, 
between the field padding and frame padding can be performed 
according to the degree of the correlation between pixels' 
in the field. 

Yet in addition, while the bit stream 200 comprises a 1 
sample identifier for each block, all the blocks can share' 
the same sample identifier. In this case, the image signal 1 
must be first divided into. fields 1 and 2, and then be f 
subjected to blocking and coding, in view of efficiency in 
waveform transform. 

Further, where all the blocks share the sample 
identifier described above, it may be provided in an image 
and at the head of a coded image signal. In this case, off 
course, the same identifier can be allocated to all the 
blocks. 

Modification 1 of Fourth Embodiment 

Figure 15 is a block diagram showing an image decoding 
apparatus according to a modification 1 of the fourth 
embodiment . 

Construction and operation of an image decoding 
apparatus 900b of this embodiment are basically similar to 
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thee of the image decoding apparatus 900a of the fourth 

embodiment in figure 14. 

in the image decoding apparatus 900b, the predicting 
section 910a of the image decoding apparatus 900a of the 
fourth embodiment is replaced by a predicting section 91 Q b 
which differs in construction from the same. 

In 2&J££ lctin * sectlon 9l0b - th « e is p'o'i— ' 

padding me*ns 920b between a motion compensating unit 911 
and an adder 906, for performing padding to a motion I 
compensated prediction image signal Pg 2 and supplying the 
resulting signal to the adder 906. The padding ^nf^foA 
includes a frame padding unit 921a, a field padding unit 921b) 
a primary switch 922 for supplying one of these padding units 
with the prediction image signal Pg 2 from the motion 
compensating unit 911 according to the sample identifier Di, 
and a secondary switch 923 for selecting one of outputs of 
the padding units according to the sample identifier Di and 
outputting a selected signal to the adder 906. 

Subsequently, operation and effects will be described. 
Motion compensation is performed to the interiaced 
image frame by frame or field by field . Tne frame fay 
motion compensation is performed by obtaining a prediction 
image signal from a frame, while the field by field motion 
compensation is performed by obtaining prediction image 
signals from the corresponding first and second fields 



separately . 

The image decoding apparatus 900b according to the 
modification 1 of the fourth embodiment provides 
effectiveness in the field by field motion compensation. m 
the image decoding apparatus 900b, switching of the primary 
switch 922 is controlled according to the sample identifier 
Di from the data analyzing unit 902, and the output of the 
motion compensating unit 911 is input to one of the frame 
padding unit 921a and the field padding unit 921b. The ' 
prediction image signal is padded by one of these padding 
units and supplied to the adder 906. * 
In this modification 1 of the fourth embodiment, since' 
padding is performed to the prediction image signal fetched 
from the frame memory 909, amount of signals to-be-padded 
in the predicting section can be reduced, as compared to a 
case where padding is performed to all the reproduced image 
signals -stored in the frame memory 909. 
Modification 2 of Fourth Embodiment 

Figure 16 is a block diagram showing an image decoding 
apparatus according to modification 2 of the fourth 
embodiment. An image decoding apparatus 900c according to 
the modification 2 of the fourth embodiment has constructions 
of the image decoding apparatus 900a of the fourth embodiment 
and the image decoding apparatus 900b of its modification ■ 
1 in combination. m other words, in addition to the 



predicting section 910a comprising the padding jji^-a'ns 920a 
of the image decoding apparatus 900a of the fourth embodiment, 
the image decoding apparatus 900c includes^auxiliary padding 
x0grrs (second padding jjie-aTis ) 920c2 provided at a next stage 
of the motion compensating unit 911, the output of which i^s 
output to the adder 906. Here it is assumed that 
constructions of the first and second padding rp^ns 920cl 
and 920c2 are identical to those of the padding nie-ans 920a 
of the fourth embodiment and the padding nxe^ns 920b of thfe 
modification 1 of the fourth embodiment, respectively. 

In the modification 2 of the fourth embodiment thuls 
constructed, the first and second padding jaoe^ms 920cl and 
920c can share padding, whereby padding can be performed in 
the predicting section with higher efficiency. 

Furthermore, a program for implementing the digital 
image padding method or the image processing apparatus shown 
in each of the embodiments is recorded in a data recording 
medium such as a floppy disc, whereby processing is performed 
with ease in an independent computer system. 

Figure 17- is a diagram showing a case where image 
processing of the digital padding method, or image coding 
or image decoding of the image processing apparatus in the 
above embodiments is performed in a computer system by the 
use of a floppy disc which contains a program of these image 
processing. 



17 ,b, shows a front appearance and . 

recording medium. Figure „ (>) ^ ^ ^ 
the floppy disc body d. The floppy disc bo dy 0 is stored 
in a case P, a„ d in a surface of the dlsc body D> plural ^ 
rrs are formed from outer to inner radius thereo( _ tra ; k 
being divided into 16 ^ ^ , 

Therefore, in the floppy disc. body D „ hich contains ^ 

program, data of the emir.™ • 

the program ls recorded in an allocated 

region of the floppy disc FD . ; 

17 ,c, shows a construction of recording an d i 
reproducing the program in/from the f lo p py disc F0 . In cas j 
of recording the program in the flo PPy disc FD, the data of 
the program is written thereto through the flo PPy disc drive 
™D from the computer system Cs. In another case of 
constructing the image decoding method in the computer system 

Cs by the program in the floppy disc FD rh« 

ppy aisc FD, the program is read 

from the floppy disc FD by the use of f ho ^ 

y cne use of the floppy disc drive 

™D and transferred to the computer system Cs. 

While description has b een given of image processing 
in a computer system b y the use of the floppy disc as the 
data recording medium, this image processing can b e a!so 
performed b y the use of an optical disc. Besides, the 
recording medium is not limited to this, and an IC card a . 
*OM cassette, and so forth, may be used so long as they can 




record a program therein. 

Industrial Av a s 1 aVvi y 

As described above, the digital image padding method, 
the image processing apparatus, and the data recording medium 
of the present invention, can increase coding efficiency i„n 
compression of an image signal, they are very useful in^ 
implementing coding or image decoding in a system which 
transmits/stores the image signal, and simultaneously they 
are suitable for compression and expansion of a moving * 
picture according to a standard such as MPEG4 . 

! 
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. . CLAIMS 

1 . A digital image padding method in which padding of pixel 
values is performed to a digital image signal forming an 
original image space cojopjiis^frg an image having an arbitrary 
shape and c^p^rSTn^ significant pixels and insignificant 
pixels, said method comprising: ' 
a pixel rearrangement s£e*Tin which plural pixels in 
the original image space are grouped as&ntf&Tg ^xtT^ 
P_r^eTrT5i"d^iHr€~to form plural small image spaces each 
comprising pixels of the same group x fTnd ~~ j 

a pixel padding in which values" of insignificant 

pixels xn each small image spac^are replaced by padding pixel 
values g en er ate d ^ a ^^U^^ 5/-^^^ 

2. The digital image padding method of claim l ; wherein in 
said pixel rearrangement sp^p t sampling is performed by 
extracting pixels every (N + l)-th (N^positive integer) 

pixel in a prescribed direction of the original image space 
(N + 1) times by using ^Ke first to ^ ( N + l)-th pixels 

as starting sample pixels^ to form (N + 1) pieces of small 

image spaces each comprising a prescribed number of pixels 

obtained by one padding. 



3. The digital image padding method of claim 2^wherein 
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operation values obtained hv r^v*~ 

ned by Performing operation on values 

of significant pixels in an n-th (n=i, 2 N + 1) small ±mage 

space are used as the padding pixel values for replacing the 
values of insignificant pixels in the n-th small image space. 

The digital image padding method of claim 2 further 
compHaes an Image space restoring step in which plural 
Pixels foWg padded small image spaces are rearranged 
according to We adapted to the prescribed rule of th'e 
grouping so that the^ixels forms a restored image space which 
has the same pixel reagent as the original image space 1 

\ 1' 

5- The digital image padding^method of claim 4 wherein 
operation values obtained by P erform>n. 9 operation on values 

of the significant pixels ln an n-th ,,»\ 2 N + 1) „ all 

.mage space are used as the padding pixel valuer replacing 
the values of the insignificant pixels in the n-th"sm H l image 
space. \_ 



6- The digital image padding method of claim ^herein in 
said pixel rearrangement sj^p, sampling is continuously 
Performed twice using ^ first and secQnd ^ ^ ^ 

starting sample pixel rowsU by extrari-i™ •> 

^ *-»y extracting pixels on every 

other pixel row in & vertical direction of the original 
image space^to form first and second small image spaces each 
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comprising a prescribed number of pixels obtained by first 
and second samplings, respectively. 

7 . A digital image padding method in which padding of pixel 

v *i„«« • * interlaced p 1 

value, 1S performed t./dj, ltll image signal fo 

" l ' lMl ln *** ! PaCe an image having an arbitrary 

shape an d c^rfslng'signif ica „ t plxels and insignifica < t 

Pixels, said method comprising^ step's 

dividing the original image space into a first smail 

image space comprising pixels on odd-numbered pixel rows in 
the original image space and a second small image space 1 
comprising pixels on even-numbered pixel rows in the original 
image space; and 

generating first padding pixel values from values of 
significant pixels in the first small image space and 
replacing values of insignificant pixels in the first small 
image space with the first padding pixel values, and 
generating second padding pixel values from values of 
significant pixels in the second small image space and 
replacing values of insignif icant pixeXs in the second ^ 
image space with the second padding pixel values. 



8. A digital image -p-a^cUng^method in which padding is 
performed to a digital image space io^Tn g an original image 
space comprising an image having an arbitrary shape and 
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Wising significant pixels and insignificant pixels , by 
replacing values of insignif ioant pixels by p ^.^ ^ 

value, determined from values of significant pixels in the 

original image space, wherein 

significant pixels which have values used for 
deter.ining.the padding pixel values are pixels other than 
Pixels adjacent to insignificant pixels to be padded ' 



9. The digital image padding method of claim 8 wherein 
significant pixels „Wh have values used for determining 
the padding pixel values are one pixel apart from the I 
insignificant pixels to\e padded. | 



10. An image processing apparatus for performing image 
coding, which comprises correlation identifying means for 
identifying correlation between pixV values of a di , ttal 
image signal and outputting a sample \enti fier accordi 
to the identification result, a coding section for perform! 
differentia coding to the digital image signal by the use 
of a prediction im a ge signal according to the sample 
identifier a nd outputting a coded image signal\ and a 
predicting section for producing the prediction image signal 
based on a decoded image signal in which the coded image signal 
has been locally decoded. 



ng 
ng 
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said coding section comprising: 

a\subtracter for computing difference between the 
digital ixmage signal and the prediction image signal as a 
difference^mage signal; 

data compressing means for compressing the difference 
image signal to produce a compressed difference image signal • 
and \ l 



a variable length encoder for performing variable 
length coding to the\ compressed difference signal, and » 
said predicting section comprising: 

data expanding mea*s for expanding the compressed J 
difference signal to product an expanded difference signalm- 
an adder for adding the X expanded difference signal to 
the prediction image signal to, produce a decoded image 
signal; 

padding means for performing' padding according to the 
sampie identifier by rearranging pixels in an image space 
formed by the decoded image signal andXreplacing values of 
insignificant pixels in the image space where the pixels have 
been rearranged with padding pixel values generated by a 
prescribed method; and 

prediction image signal producing means foV producing 
the prediction image signal from the padded decoded image 
signal according to the sample identifier. 
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The image processing apparatus of claim 10 wherein said 
coding section further comprises padding means for 

PeIf °T' Paddi " 9 t0 the difference image signai according 
to the sW e i dentifier by rearranging pixels in an . mage 

space forme^by the difference image signai and replacing 
values of insUfica.t pixels in the image space where the 
Pixels have beenWarranged with padding pixel values 
generated by a pres\ibed method, and said data compressing 
means performs information compression to the padded 
difference image sig„al\o produce a compressed difference 
signal. \ j 

\ I 

12. An image processing apparatus for performing image 
decoding, which comprises a reproducing section for 
reproducing a coded image signal in\hich a digital image 
signal has been coded b/the use of a prediction image signal 
of the digital image signal and out P uttin g \ reproduced image 
signal, and a predicting section for producing, the prediction 
image signal from the reproduced image signet, 
said reproducing section comprising: \ 
a data analyzing unit for analyzing the co^d image 
signal to extract a compressed difference signal \f the 
digital image signal and a sample identifier indicating 
correlation between pixel values of the digital image signal, ■ 
from the coded image signal; 
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data expanding means for expanding the compressed 
difference signal to produce an expanded image signal; and 

\ adder f ° r addin 9 th * expanded difference signal to 
the prediction signal to produce the reproduced image signal, 

and \ 

in said predicting section, 

using at least one of the reproduced image signal and 
the prediction ima^e signai obtained from the reproduced 
image signal as a signal to be padded, padding is performed 
to the signal to be padded according to the sampie identifier, 
by rearranging pixels in ^i m a g e space formed by the signa'l 
to be padded and replacing \alues of insignificant pixelt 
in the image space where the pixels have been rearranged with 
padding pixel values generated \ a prescribed method. 

\ 
\ 

13. The image processing apparatus ofVlaim 12 wherein said 
predicting section further comprises padding means for 
performing padding to the reproduced image Signal frame by 
frame or field by field according to the sampleidentif ier, 
and a frame memory for storing a padded reproduced image 
signal in which the reproduced image signal has beeVpadded, 
and outputs the padded reproduced image signal stored in the 
frame memory to the reproducing section as the prediction 
image signal . 
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Vl • The image processing apparatus of claim 12 wherein said 
Predicting section further comprises a frame memory for 
"°V Reduced i",age signal, and padding means for 

performing padding to the prediction image signal extracted 
from the reproduced image signals stored in the frame memory 
frame by fr\me or field by field according to the sample 
identifier, and outputs the padded prediction image signa'l 
to the reproducing section. 

15. >«^1 imaged™,, apparatus for performing image 
decoding, „ hich comprises a reproducing section for I 
reproducing a coded ima g e\gnal in which a digital imagi 
signal has been coded by the us\of a prediction image signal 
of the digital image signal and ouWing a reproduced image 
signal, and a predicting section ( „Wi„, the prediction 
image signal from the reproduced image signal, 
said reproducing section comprising: 

a data analyzing unit for analyzing\he coded image 
signal to extract a compressed difference signal of the 
digital image and a sample identifier indicating Correlation 
between pixel values of the digital image signal,' irom the 

coded image signal; 

data expanding means for expanding the compressed 
difference signal to produce an expanded difference signal; . 

and 
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an adder for adding the expanded difference signal to 
the^ediction image signal to produce the reproduced image 
signal, 

said pr>d^cting section comprising: 

padding mea\for performing padding to the reproduced 
image signal accordin^he sample identifier, by replacing 
values of insignificant pifc^a in an image space formed ' y 
the reproduced image signal wiHoadding pixel values 
generated from values of significance!, other than pixel's 
adjacent to the insignificant pixel; \ 

rearrangement means for rearranging pixels in an i.aje 
space formed by the padded reproduced image signaYa.ccordin!, 
to the sample identifier. 



16. A data recording medium for storing a program w^ff 
maJces a computer perform padding of pixel valuesu^tal 
image signal forming an original image spac^otprising an 
image having an arbitrary shape and casing significant 

pixels and insignificant pixels,/^ 

said program making th^mputer perform padding of 

Pixel values according said digital padding method of 

claim 1. 



17. Ad^a recording medium for storing a program which 
makes/cornputer perform padding of pixel values to a digital 
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image sigl^f^rmi^g an original image space comprising an 
image having an ar^^shap e and comprising significant 
pixels and insignificant pixel^ 

said program making the computer^™ p adding of 
Pixel values according to said digital image padding method 
of claim 7. ' 

) 

v8. A data recording medium for storing a program which 
mak^s a computer perform padding of pixel values to a digita* 
image Wial forming an original image space comprising ah 
image ha\ng an arbitrary shape and comprising significant} 
pixels and\nsignif icant pixels, | 
said program making the computer perform padding of 
Pixel values acceding to said digital image padding method 
of claim 8 . 

19. A data recording mekm for storing a program which 
makes a computer perform imag^ signal coding, 

said program making the confer perform differential 
coding to a digital image signal b\the image processing 
apparatus of claim 10. 



20. A data recording medium for storing a program which 
makes a computer perform image signal decoding\ 

said program making the computer perform differential 



ABSTRACT 



In a digits image padding method according to 
present invention, as shown in fig ure 2 , every othgr 

,B " ° rl9inal lma9e -Wjl* corresponding to an input 
coital ima9 e signal are extracted in the horizontal 
direction, and every other pixels are extracted in the 
verticai direction, the extracted plural plxels ar fe 

rearranged to form four «.= n ■ ^ 

rour small image spaces 4£r^4jJ<, and 

Pixels values in these small image spaces are padded. I 
in the digital image pa dd in g method thus constructed! 
padding of pixel values is performed to plural small image 
spaces where there is high correlation of pixel values 
between adjacent scanning lines rather than to an original 
image space where there is low correlation of pixel values 
between -adjacent scanning lines. That is, in the padded 
small image spaces, there is high correlation of the pixel 
values. As a result, it is possible to avoid introduction 
of high-frequency components into the digital image signal 
of the interlaced image, and hence, differential coding or 
differential decoding can b e performed to the interlaced 
-age or the li Ke with high coding efficiency while 
suppressing degradation of coding efficiency resulting from ■ 
padding. 
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